When should I think about MLOps?

This video and post are part of a One Dev Question series on MLOps - DevOps for Machine Learning. See the full video playlist here, and the rest of the blog posts here.

Learning a new way of working can be challenging. But it's important to apply MLOps practices and techniques at the right time. That time is as early as possible!

There's a common anti-pattern I see in software projects, and I've seen it for years. Leaving the deployment of your application until there's something meaningful to deploy.

On the surface, it sounds like a reasonable decision. After all, time is limited, and if the application doesn't even do the minimum yet, surely spending time trying to deploy it is a waste of cycles? Better to work on features, right?

The problem with this argument is that by deferring deployment to the end, you're building up software that's more and more complex, and therefore harder and harder to deploy. In many cases, delaying means you'll never get a great result because the system won't have been designed for easy deployment.

It's similar to software testing. If you're doing test-driven development (TDD) or even just writing tests when you write the code, your code necessarily becomes testable. If write some code and can't write a test for it, you immediately know it's going to be hard to diagnose if there's a problem.

Similarly, if you only think about how you're going to deploy your software once it's large and complex, you miss the opportunity to optimize for a fast cadence. Being able to continuously and quickly deploy small changes is one of the goals of DevOps, and leads to much better software delivery outcomes.

Blurred cars moving fast on a road. Photo by Ruiyang Zhang from Pexels

So how does this relate to ML projects? In exactly the same way.

If you only think about how you're going to train, package, and deploy your models towards the end of your project, it's going to be much harder to do.

For example, let's imagine you have a team of data scientists working on a recommendation engine for your product catalogue. The ultimate goal is to use this model on your company's ecommerce site to increase sales.

You and your team work relatively independently with your own Jupyter notebooks and database extracts. Maybe you pass notebooks to each other by email and have regular meetings and pairing sessions to make sure you're all working together. Training is performed by running your notebooks on one of a few shared Data Science Virtual Machines in the cloud.

At some point, you get a solution you think will work. It's producing good recommendations - at least from the data extracts you've been using! You have a great spreadsheet with plenty of examples and proof, pulled right from your notebook.

So, time to move to the next step. But how do you do that?

How do you get from those few cells in Jupyter to a production ecommerce site?
How can the web application use your model?
Can your team reproduce your experiment and work on tweaking it for better results?
How will you know if the model is performing in the real world?

This can be the hardest part. Modifying what you've done in order to create something you can operationalize can be really tricky. If you don't consider it until the end of your project, you might be in for a world of frustration and rework!

A woman at a laptop looking frustrated. Photo by Andrea Piacquadio from Pexels

In my consulting days, we had a process for new projects. Right at the start, we'd build a full end-to-end pipeline for the absolute minimum application. For example, if it was a web app, it might literally be a "Hello World". We'd have all the code in source control, a continuous integration build that ran an empty test suite, and continuous deployment to "test", "staging", then "production".

Even though very little code would survive to the end, and the CI/CD processes built, tested, and deployed almost nothing to servers pretending to be test/staging/production, it meant that we had our pipeline and processes already working. We got into the habit of using it, tweaking it, and trusting it.

Importantly, when we added a feature, a dependency, or something that required a change to the pipeline, it was just that - a change. Not something we have to work out in 6 months, or an unexpected, "oh I forgot about that" surprise right before project delivery date. Just like code, small frequent changes are easier to manage than big infrequent ones.

Small huts on the beach. Photo by Taryn Elliott from Pexels

DevOps is similar to design, accessibility, testing, security, and a host of other practices in that it should be considered right from the start. It's very difficult to get a good result if you only think about these things once most of the work is done!

The same goes for MLOps.

The best time to think about MLOps is day one. The second best time is now.