What Not to Automate Fully

An important part of DevPro's November focus on automating and scaling management of devices, users, and software across your business is selection of what not to automate. As I already emphasized in first raising this topic, engineering is all about trade-offs, and imbalanced automation costs at least time and money better used elsewhere.

You can capture most of the value of many projects with a partial automation: while you don't end up with a single button you can push that completely solves a particular problem, you have a write-up or checklist that reliably guides any teammember through all necessary steps. From a managerial perspective, for instance, it matters little whether we have a perfected piece of software that performs an operation, or a definite process that humans execute to achieve the result: either answer is adequate at a business level. When combinatorics of a difficult sequence or delicate security configuration or network variability make it tough to analyze and (especially!) test a complete automation, write up everything the next technician will need to know and declare victory. While a human-readable script doesn't immediately scale to thousands of servers, a lot of what we do only needs to apply to a handful of machines at a time.

False Step

Too many sysads believe that job security is a reason not to automate. More precisely, it is a reason, but a mistaken one. Yes, we've all seen engineers and techs whose career model is to hoard information and make themselves as apparently irreplaceable as possible. In this scheme, of course, automation is just a threat.

Not to automate, though, when it's best for the organization as a whole, is professional misconduct. While it can seem like a safe and even lucrative choice for a while, savvy employers notice how it makes them vulnerable. Smart companies shed workers with that attitude as fast as they can.

Often even more important is the damage the practitioner does himself. To corrode ones own integrity and ability to advance through a deliberate choice not to automate a workflow sensibly is a stiff price to pay.

I'm not overestimating the loyalty of employers of the twenty-first century. I recognize how quick nearly all are to slash payroll when they can. The best way for the individual worker to armor against such a dismal outcome, though, is to improve operations continually and gain a reputation for exceeding expectations, not to try to hold the organization hostage to the secrets you know.

Payback

A good reason not to automate a process fully is that the return won't be sufficient. That doesn't necessarily mean the process is unimportant, just that it's infrequent or predictable or inexpensive enough that a "manual" approach is feasible. An annual budget festival, for instance, might strain your existing servers' capacity: you know that the operations staff will need to be available to clean up filesystems, ensure network availability, and migrate lower-priority tasks away. As a once-a-year event, though, it might be easier simply to schedule a few extra shifts for operations for one month, than to buy the capacity that would permit the planners to run "hands-off". While variable capacity is much of the appeal of "the cloud", whether internal or external, the cost of the existing manual approach might be hard to beat.

A few other positive reasons for partial automation have to do with a choice for human scale. We're all familiar with authentication processes that introduce random and/or long delays in order to thwart brute-force cracking. Many organizations deliberately require customers in particular categories to contact a live human, either to qualify the customer more accurately, to brand the organization's personal touch, or as another element of security. Moreover, while I labeled it "misconduct" not to automate when analysis concludes that automation is justified, there's nothing wrong with making the cost of an automation project apparent before embarking on it. When the burden of a particular manual operation falls mainly on one particular department, began to automate that operation only when the department that will benefit has budgeted a fair share of the cost of automation. Has someone from the executive suite requested a particular IT shortcut or enhancement to streamline his or her own habits? Feel free to use your own judgment on that one; I hope you feel safe, though, answering that that request will need to go through whatever review and prioritization is customary for IT.

More broadly, recognize that perfection is unattainable, and don't let an abstract notion of perfection interfere with the real progress you can achieve. Like Dr. Rebecca Parsons, CTO of Thoughtworks, I'm a fan of software that works. Even when it's known to be dirty, it still can be useful. Parsons makes the point that the wiser motivation for refactoring is not to achieve an academic level of "purity" of our code, but to support the future of our organization. If a manual workflow is congesting operations and distorting decision-making, it's a candidate for automation; if everyone is comfortable doing a few pieces by hand, though, and the workflow is secondary to the organization's future growth, then let it ride, and put your efforts where they'll pay off more dramatically.