When, Where, and How to Apply DevOps Automation (tl;dr Everywhere)

Are your deployments boring? If they’re not, my bet is that you sure wish they were.  

Why is a boring deployment a good thing? By boring, I really mean uneventful...deployments that take seconds, not hours. Boring, uneventful deployments happen with a few clicks, rather than by following every step in a long (and usually tedious) Confluence doc. 

When things aren’t automated, you usually get those eventful deployments. Of course, that’s the last thing you want.

However, when you leverage DevOps automation correctly, you streamline your entire software development lifecycle. Developers can get code out faster because they’re not manually clicking through dozens of steps they have to follow.

When, where, and how does that actually happen? 

“The Best Deployment is a Boring Deployment”

While this is one of the key phrases we use at Sketch, we acknowledge that it’s not just a Sketch thing. It’s still something we swear by, whether we originated it or not. 

What you want, and what we always aim for, are boring, uneventful deployments.

What do we mean by that? An eventful deployment is a sign that something has gone terribly wrong. Eventful deployments usually require getting an on-call developer or admin involved to put out whatever fire happened. Those errors are typically associated with  manual, labor-intensive steps during the deployment process. 

We’ve worked with clients where deployments take hours simply because they’re overly reliant on a giant check-list of steps that a developer has to follow. When something is fully automated, it’s repeatable! That deployment will work every single time as you promote your code from your lower environment into the QA environment. There’s no reason you shouldn’t be able to use that exact same automation process to get your code from QA to staging, from staging to production.

How can we make sure all of your environments are stable enough to enable those critical automated phases? That’s when we talk about environment parity.

What is Environment Parity? 

If your environments are different from each other, automation could invite instability. 

If your environments were set up in a manual fashion, there’s always a chance (and granted, this is usually completely unintended) that someone mis-fingered something in one of the other environments. Maybe you have a different version of Java installed in production that doesn’t exist in a lower environment. Or, maybe the name of an environment variable has a typo resulting in your application totally blowing up.

Automated and manual testing could easily prove-out success in a given lower environment. But when it’s time to launch something in the next-higher environment (i.e., production), your code may suddenly stop working the way you expected it to, because you had an error in one of those manual steps somewhere along the way.

So, if we’re talking about “where” DevOps automation should happen, it’s most effective in an environment where each stage is in line with the next. Typically, we see the most success with that in either a public cloud environment (AWS, Azure, or Google) or Kubernetes.

DevOps Deployments with Kubernetes

Kubernetes is a cloud-agnostic, open-source platform that offers many of the same features you find in public cloud environments. 

When you’re using principles of Infrastructure-as-Code, all of your environments should be at parity. This further enables you to automate not only the deployment of your code but your actual infrastructure itself. Once your infrastructure is automated and you have environment parity between all of your stages, that is precisely when and where you should be leveraging DevOps automation for deploying your code.

So...HOW Do We Deploy DevOps Automation?

“How” we apply DevOps automation is always a bit relative. But I’ll do my best to talk to the product owner who is frustrated that deployments are buggy, taking too long, and just generally exhausting or frustrating.

One of the biggest gifts any team can give themselves is a continuous integration (CI) tool/platform like Jenkins. Jenkins is probably the most well-known standard in the industry. It also satisfied the compliance standards that a lot of regulation-heavy industries (healthcare and finance are two examples) demand.

When you set up your CI instance to execute this level of automation, developers aren’t doing the actual deployment. Your CI tool is.

How does this help?

1. Everything is automated, which should give us that uneventful or boring deployment we aim for.
2. It keeps the developers’ fingers out of the secured environments, increasing your deployment efficiency without sacrificing or compromising compliance...you have maintained your separation-of-duties.

CI isn’t human, which is precisely the point. It’s far, far faster and less error-prone. Leveraging continuous integration still requires expertise when it comes to writing the code and scripts to do those repeatable deploys.

When Should This Happen?

From the beginning, you should be aiming for this level of automation and parity when you're building the application from scratch. Not only should you be thinking about which features you want to implement, but you should also consider how you’re going to get it in front of the end users.

Of course, this is advice that would have been useful before you launched your app. What if if you’re already well past that point? 

You can retroactively script deployments, and there’s nothing wrong with doing that. We’ve worked with clients who have long internal Confluence docs or internal wikis that have 30-plus steps that could easily have been put into a script. 99.9% of the time, there’s not a single reason we could find where all of those steps had to remain manual.

When there is a manual step (which, in my humble opinion, there never should be, but let’s say there is), you can still script everything before the manual step, and then script everything after the manual step. That way, the only closed bottleneck is that one manual area. Eventually,, creativity takes over and we discover a way to automate that manual step while we’re automating the rest of the deployment. 

Example: A team of developers we worked with was hung up on a step they were hesitant to abandon. Previously, they’d relied on a manual process that involved moving files via FTP.

The simple solution? A command line tool that does the exact same thing. Even in that instance, we were able to move through a manual sequence and automate it with existing tools.

Automation may not work if you’re still bogged down with outdated manual processes. The key, in those cases, is to improve your infrastructure (clean it all up or move it to the cloud) so you can automate your production environment. Of course, we can help with both of those things.

What Happens When Code Needs to Go in Front of a Review Board?

Compliance is often a sticking point in automated deployments. Before new code goes live, it sometimes does have to be reviewed by a compliance board (this is usually particular to heavily regulated industries, like healthcare, finance, aerospace, etc.). If there is a regulatory process the code has to satisfy before it can go into production, there are plugins  in most continuous integration platforms that you can set up in your pipeline.

The process is also relatively straightforward. We place control of the “go” button to those regulatory or compliance bodies. Once it’s clicked and the process is completed, Jenkins can continue its automated boring deployment all the way to production.

Old Habits and Cultural Mismatches 

There are two really common scenarios we run into that share a common theme: Outdated practices and bad technology integration. While they are two very different issues, they can still result in costly, time-consuming deployments.

Old Habits Die Hard

Developers are as prone to old habits as anyone. People come and go, but processes seem to live forever. You could also be working on a team with developers that are using the same tools they used from day one...20 years ago. New people often don’t have time to rebuild or improve systems because the reality of day-to-day work can unfortunately get in the way of systemic improvements.

In any case, old, outdated systems and methods become ingrained in those organizations. The minute you recognize you have aging systems and infrastructure, that may be an ideal time for us to come in and set up an automated DevOps pipeline.

Your product owners and their teams shouldn’t be spending 30 minutes to an hour just to see code in a new QA environment. That can (and should) take 90 seconds...tops. Removing that burden from their shoulders frees them up to do what they should be doing: innovating creative and exciting solutions for you and your customers.

Messy Mergers

When larger companies snap up smaller companies, systems can get mismanaged pretty quickly. Larger companies can often be guilty of forcing the upstart they just purchased to conform to “their” way of doing things, even if the reason that company was targeted for acquisition is because the way they did things was better.

When the old ownership or engineering talent are no longer in the driver’s seat of their own dev environments, suddenly everything slows down. The corporate machine prioritizes its talent and culture, maybe that smaller team gets reassigned to other teams…you know the drill. Many of us have lived through it, right?

The bigger the company’s footprint, the bigger the potential for blind spots. Just because you have a good product doesn’t mean you have a good process. Sketch has come into many environments to help clean up integrations and get the teams working together, saving everyone a lot of time, money, and frustration.

What’s the ROI of DevOps Automation?

How much can automation save you from a budget standpoint? 

Think of how much you pay your engineers. Consider how long a deployment takes them today (and how much that costs based on your previous answer). Now consider how much you could save by turning their deployments into boring, dull, repeatable ones. Granted, before the cloud, when environments were different, we used to get calls at 2 a.m. just to get the deployment done by the time the office was opened. One of the jokes that I make is that Jenkins can do all of this in its, and your, sleep.

I still have yet to talk to any product team that is resistant to saving both time and money. On that note, it doesn’t cost us—or you—anything to talk.