This video and post are part of a One Dev Question series on MLOps - DevOps for Machine Learning. See the full video playlist here, and the rest of the blog posts here.
It might seem that changing the way you work doesn't necessarily improve the work you produce. However, by focusing on the right things, the final result can be improved simply because you're spending the time on the product rather than peripheral issues.
In a previous post, I talked about how implementing MLOps meant that all the peripheral tasks around actually creating a predictive model fall away, allowing your data scientists to focus on creating the best solution.
Let experts focus on what they're best at.
In short, if all of the effort of the machine learning experts and data scientists is going towards producing better models, there's much less wasted effort. It's focused on the end goal.
A good MLOps process can take care of most of the heavy listing when it comes to things like managing engineering environments, managing production environments, versioning and collaborating on code, executing experiments and training runs, testing, validating, and versioning predictive models, packaging those models into a deployable form, and deploying them in safe ways.
Gene Kim talks about "focus, flow, and joy" as the second of his DevOps ideals. What this means in practice is that the more you can focus on a single task, the better the outcome!
Just as important is being able to know what is valuable. DevOps is all about delivering value to end users, but without measuring what is and isn't valuable, you can't know whether the work you're putting in is positive change. This also goes for MLOps.
For example, your data scientists may have created a better predictive model by adding extra layers and nodes to the neural network. It now makes a correct prediction 95% of the time instead of 90%. Great, right?
But this extra 5% may not be valuable. If that model takes twice as long to make predictions, that 5% gained by users may be more than offset by the pain of waiting for a response. Where the model was supposed to make (for example) filling out a form faster, it might now be faster just to type.
If there was no (or slow) feedback telling the team that performance needed to be a priority, there's a good chance all that effort is being poured into eking out another couple of percentage points of accuracy on a model that's too slow to use.
So just as we do with agile software development, it's important to look at where the effort should go, and objectively evaluate (as much as possible) whether a change has provided value or not.
So an organisation with good MLOps practices can ultimately produce better predictive models in two ways:
- They let the machine learning experts and data scientists focus on what they're good at - creating better models
- They encourage fast feedback so that focus can be directed to the right place