DevOps for Brownfields - Process

In my last post, I claimed that while DevOps was all about culture, that's not very helpful when it comes to getting started. You can't force a culture change.

However, there are things you can do - even as a relatively powerless developer - that can set you on the right path.

What are we trying to do?

Ultimately "doing DevOps" comes down to one key goal:

Improve your cycle time

A faster cycle time means changes spend less time in the pipeline. This has a few advantages. If a bug is introduced, it's very likely that the team that worked on that code did so recently. It's fresh in their minds and therefore easier to fix. Second, it provides for far more agility. Code that's waiting to be deployed for months may not be relevant by the time it gets to production. And if there's a new opportunity, code can be written and deployed with very little delay.

A faster cycle time also means smaller changes being released to production more frequently. Smaller changes means less risk, and if something does go wrong, it's far easier to pinpoint and fix the problem. In many cases, it's faster and safer to fix the issue and deploy than it is to roll back - which is a good thing.

What can we speed up?

First, let's define this "cycle". Keeping things high-level, here's what a cycle looks like:

  1. Define a feature or requirement
  2. Write code and tests and push to source control
  3. Build the code and run tests
  4. Deploy to a pre-production environment and run tests
  5. Deploy to production
  6. Monitor what happens in production
  7. Use the results from 6. to guide 1.

The faster these steps can occur, the shorter your cycle time will be.

The team should focus on items 1, 2, and 7. Deciding what to create and creating it is what people are good at.

Items 3 - 6 should happen automatically. Repetitive, precise tasks are not what people are good at. People are slow, and they make mistakes. You'll find you simply can't be fast without effective tooling that does the work for you.

What are the practical steps?

Let's look at items 3-6 one by one, starting with item 3 - building and running tests.

Set up a Build Server

You absolutely, 100%, without a doubt need a build server.

Configuring a build server is the easiest and most beneficial thing you can do to improve your process.

I personally love the new Team Build available (for free) in Visual Studio Team Services - it'll build anything you can throw at it in any language, and you don't have to use VSTS for anything else if you don't want!

Once you have a build set up, configure it to build on every commit - a Continuous Integration build. A build server that compiles your code and runs tests on every code change provides two main advantages:

  1. You'll know immediately if a code change has broken your software
  2. You'll always have a deployable build ready to go

Deployment

Steps 4 and 5. Deployment.

If you want to improve your cycle time, you'll need tooling for this. Going through a printed document checking off items as you manually deploy to multiple machines just isn't going to cut it!

Again, VSTS has an option in Release Management, but I'm confident in calling this one for Octopus Deploy. I'm naturally biased because I work there (Disclaimer!), but the reason I work there is because I love the product. Find an Octopus user, and ask them how they feel about it.

Conveniently, it works really well with the VSTS build system mentioned above. Just add and configure the Octopus steps available in the marketplace extension.

Deploy the same bits to pre-prod and production, and deploy them the same way.

It's really, really important that each time you deploy a release candidate to an environment, you deploy the same artifact built by your build server above (don't rebuild), and deploy it in exactly the same way.

Doing so gives you two advantages:

  1. You're testing the same bits that will go into production
  2. You're testing the method by which those bits get to production

Importantly, this means externalising all your configuration. You shouldn't have environment-specific settings in compiled code - rather, keep it in config files or databases or even external services. I'll elaborate on this in a future blog post.

Monitor Production

If you're deploying to the web, there's no longer a reason to be surprised by a customer telling you about an issue in production.

If you're searching for clues in the logs, by definition you're a step behind

As developers, we've been writing to error logs for decades, and clever tools like Seq can make them extremely easy to analyse. However, there are countless tools that let you get ahead of the problem.

If you're a .NET developer writing web apps, Application Insights literally takes minutes to add to your application, and for relatively small amounts of data, it's free. With very little setup, it'll alert you to performance problems, exceptions, and all sorts of other things you want to know about before your customers do.

Summary

Setting up these three things - a build server for CI, an automated deployment pipeline for CD, and monitoring for Production - will cover most of what it means to "do DevOps", at least from an automation perspective.

These items are a great start, but constant introspection and improvement are just as important. It's not enough just to have these in place, just like it's not enough to have a standup and sprints if you're "doing Agile". You need to continue to refine and adjust so the process works as well as it can.

If you get your process tooling right, your team can focus on what they're best at - writing code.

Damian Brady

I'm an Australian developer, speaker, and author specialising in DevOps, MLOps, developer process, and software architecture. I love Azure DevOps, GitHub Actions, and reducing process waste.

--