Advice for the Lazy Administrator
Steps you can take to avoid late nights and system frights.
I always liked the saying that "a lazy computer operator is a good computer operator." Many operators are always looking for ways to practically automate themselves out of a job. For them, the reasoning goes: "why should we be manually doing things, if the machine can do them instead?"
A few hours spent writing a script or tool can pay for itself very quickly by freeing up the operator's time to perform other tasks. If you set up your script and crontab entry correctly, you can let your machine remember to take care of the mundane tasks that come up while you focus on more important things, with no more forgetting to run an important report or job. Sadly, even the best operator with the most amazing scripts and training will need help sometimes, at which point it's time for the page out.
In our jobs as system administrators, we know we're going to get called out during off hours to work on issues. File systems fill up, other support teams forget their passwords or lock themselves out of their accounts at 2 a.m., hardware breaks, applications crash. As much as we would love to see a lights out data center where no humans ever touch machines that take care of themselves, the reality is that someone needs to be able to fix things when they go wrong.
We hate the late night calls, but we cope with them the best we can. Hopefully management appreciates the fact that many of us have families and lives outside of work. We are not machines, or part of the data center. We can't be expected to function all day at work, then all night after getting called out. It's difficult to get back to sleep after getting called out, and it impacts our performance on the job the day after we are called or, worse, it ruins our weekends. However, our expertise and knowledge are required to keep the business running smoothly with a minimum of outages, which is all factored into our salaries.
I have seen different methods used, but it's basically the same. Each person on the team gets assigned a week at a time, with some jockeying around to try to schedule our on-call weeks to avoid holidays, and usually people can work it all out at the team level. In one example, I even saw cash exchanging hands to ensure that one individual was able to skip his week. Whatever method is used, the next question revolves around how long you're on call. Is it 5 p.m. - 8 a.m. M-F and all day Saturday and Sunday? Is it 24 x 7 Monday through Monday? Does the pager or cell phone get handed off on a Wednesday? Do we use individual cell phones or a team cell phone? They are all answers to the same question, and you have to find the right balance for the number of calls you deal with off-shift and the on-call workload during the day.
Search our new 2013 Buyer's Guide.
Tips and Techniques | Overcome eight key challenges and reduce costs
E-Newsletter | Steps you can take to avoid late nights and system frights.
Technical Corner | A mainframe systems programmer must address high-level issues when installing a data warehouse