MLOps Blog

MLOps Engineer and What You Need to Become One?

6 min
27th September, 2023

So you want to become an MLOps Engineer (or maybe hire one). Ok, are you sure you know what you are getting into?

An MLOps Engineer sits between Machine Learning, Software/Data Engineering, and DevOps, combining good practices from all to enable successful deployment and management of machine learning models in prod environments.

What is MLOps? (Machine Learning, DevOps, Data Engineering)
What is MLOps?

In short, if you:

  • 1 know machine learning algorithms and frameworks – TensorFlow, PyTorch
  • 2 can code in Python/Java, 
  • 3 deploy your code on AWS/GCP/Azure, 
  • 4 have a good grasp of CI/CD pipelines, IaC (Infrastructure-as-code) tools (like Terraform, CloudFormation),
  • 5 have experience in working with relational and non-relational databases, data warehousing, and data streaming frameworks (think Apache Kafka/Spark/SQL)
  • 6 familiar with concepts like firewalls, encryption, VPNs, and secure data transfer.
  • 7 have worked on Logging tools like Prometheus, and ELK Stack

then you should demand a raise! 

Jokes aside but knowing at least 70% of these topics would get you the role you wanted and a career in MLOps.

In the ML stack of things, MLOps engineer sits towards the far right end, starting with 

  1. Data Scientist: who formulate solutions, work with the stakeholders and design data-driven solutions to problems at hand.
  2. ML Engineers/Data Engineer: They work their charm on the analysis and models developed by Data Scientists to more prod-ready versions with good code practices and scalable products.
  3. MLOps Engineer: While there is a fine line b/w them and ML Engineers, MLOps engineers sits on the infrastructure side of things rather than development. They work with ML Engineers to get the pipelines up and running, setting up CI/CD, firewalls, and tracking for the longevity of machine learning models.

What is MLOps Engineering?

MLOps is the discipline of applying DevOps ideas to machine learning systems. It facilitates the creation and deployment of machine learning models in large data science projects. In larger initiatives, operational elements require just as much brainpower to put those models into prod as the dev elements.

  • MLOps engineers deploy, manage, and optimize machine learning models in production environments, ensuring smooth integration and efficient operations. 
  • MLOps Engineers take a data scientist’s model and make it accessible to the software that utilizes it. 

To have more clarity around this, let’s ask Dmitry Goryunov, MLOps Engineer at Deepset, about what on earth does an MLOps engineer do?

Oh, it depends. When you work at a company like Zalando, your work makes data scientists productive. So you work closely with data scientists to understand what they are about to do, to define their requirements. Then you try to fit their requirements inside the company’s infrastructure, build the data pipelines, deploy the servers to host the models, organize monitoring and continuous integration. 

Now that I work for Deepset, I try to generalize my previous experience. Now it is more about building a convenient toolbox so that the data scientists who work with semantic search and question-answering systems can train, evaluate, and deploy those systems just by clicking a few buttons.

So, on a very broader scale, a BAU for an MLOps Engineer would be something like:

  • Checking deployment pipelines for machine learning models.
  • Review Code changes and pull requests from the data science team.
  • Triggers CI/CD pipelines after code approvals.
  • Monitors pipelines and ensures all tests pass and model artifacts are generated/stored correctly.
  • Deploys updated models to prod after pipeline completion.
  • Works closely with the software engineering and DevOps team to ensure smooth integration.
  • Containerize models using Docker and deploy on cloud platforms (like AWS/GCP/Azure).
  • Set up monitoring tools to track various metrics like response time, error rates, and resource utilization.
  • Establish alerts and notifications to quickly detect anomalies or deviations from expected behavior.
  • Analyze monitoring data, log, files, and system metrics.
  • Collaborate with the data science team to develop updated pipelines to cover any faults.
  • Documenting and troubleshoots, changes, and optimization.

This brings us to how MLOps engineers are different from Data Scientists, Software Engineers, Data Engineers, and ML Engineers. 

MLOps Engineers vs. Data Scientists 

Data Scientists specialize in finding and applying the optimum machine learning model to handle business challenges. They experiment with different algorithms, fine-tune their hyperparameters, and then assess and corroborate their results using a range of standards. 

While Data Scientists drive the development of machine learning models, MLOps Engineers enable their deployment, integration, and ongoing management, bridging the gap between data science and operations to ensure the efficient and effective use of machine learning models in real-world applications.

MLOps Engineers vs. Software Engineers

Software Engineers focus on access control, use data gathering, cross-platform integration, and hosting, encompassing various aspects such as architecture, coding, testing, and debugging. 

While Software Engineer handles the broader software development lifecycle, MLOps Engineer brings their expertise in machine learning and operations to effectively deploy and manage machine learning models within software systems.

MLOps vs. Data Engineers

MLOps Engineers primarily focus on the deployment, management, and monitoring of machine learning models in production, bridging the gap between data science and operations.

On the other hand, Data Engineers specialize in designing, building, and maintaining data pipelines and infrastructure for efficient and reliable data processing and storage. 

While there is overlap in some areas, MLOps Engineers concentrate on ML model deployment and management, while Data Engineers focus on data infrastructure and pipeline development.

MLOps Engineers vs. ML Engineers

ML Engineers enable model deployment automation to production systems. The amount of automation varies within the organization. They take a data scientist’s model and make it accessible to the software that utilizes it. 

Machine learning models are commonly built, tested, and validated using Jupyter notebooks or script files. However, software developers want machine learning models to be available through callable APIs like REST.

ML engineers may sit on platform teams as well as individual ML dev teams depending on the size of the company and the requirements of their machine learning models. 

Here’s what Amy Bachir, Senior MLOps Engineer at Interos Inc., said when questioned, “Is there any difference b/w an ML Engineer and an MLOps Engineer?”

Yes! Absolutely! In my opinion, ML Engineers build and retrain machine learning models. MLOps Engineers enable the ML Engineers. MLOps Engineers build and maintain a platform to enable the development and deployment of machine learning models. They typically do that through standardization, automation, and monitoring. MLOps Engineers reiterate the platform and processes to make the machine learning model development and deployment quicker, more reliable, reproducible, and efficient.

To be graphical:

Different types of data professionals
Types of data professionals | Modified based on: source

There is a close resemblance of an MLOps engineer to an ML Engineer, but the role difference on the infrastructure side gets prominent. Above is a good overview of responsibilities.

MLOps Engineer role requirements

If you’re an aspiring professional ready to adopt an MLOps Engineer role, below is what you need to do that job and what it’s going to look like (in broad strokes).

MLOps Engineer skills and tools

Note: These requirements are quite broad and may not represent any Job description. MLOps-related jobs can come from a wide spectrum of industries and may contain a subset of these requirements.

  • Deep quantitative/programming background with a degree (Bachelor’s, Master’s, or Ph.D.) in a highly analytical discipline, like Statistics, Economics, Computer Science, Mathematics, Operations Research, etc.
  • Total of 3-6 years of experience in managing machine learning projects end-to-end, with the last 18 months focused on MLOps.
  • Monitoring Build & Production systems using automated monitoring and alarm tools.
  • Knowledge of machine learning frameworks: TensorFlow, PyTorch, Keras, Scikit-Learn.
  • Experience with MLOps tools such as ModelDB, Kubeflow, Pachyderm, and Data Version Control (DVC).
  • Experience in supporting model builds and model deployment for IDE-based models and autoML tools, experiment tracking, model management, version tracking & model training (Dataiku, Datarobot, Kubeflow, MLflow, neptune.ai), model hyperparameter optimization, model evaluation, and explainability (SHAP, Tensorboard).

Aside

neptune.ai is an experiment tracker for ML teams that struggle with debugging and reproducing experiments, sharing results, and messy model handover.

It offers a single place to track, compare, store, and collaborate on experiments so that Data Scientists can develop production-ready models faster and ML Engineers can access model artifacts instantly in order to deploy them to production.

See in app

MLOps Engineer job responsibilities

  • Deploying and operationalizing MLOps, in particular, implementing:
  • Model hyperparameter optimization
  • Model evaluation and explainability
  • Model training and automated retraining
  • Model workflows from onboarding, operations to decommissioning
  • Model version tracking & governance
  • Data archival & version management
  • Model and drift monitoring
  • Creating and using benchmarks, metrics, and monitoring to measure and improve services.
  • Providing best practices and executing POC for automated and efficient model operations at scale.
  • Designing and developing scalable MLOps frameworks to support models based on client requirements.
  • Being the MLOps expert for the sales team, providing technical design solutions to support RFPs.

MLOps Engineers work closely with Data Scientists and Data Engineers in the Data Science Team from the start of the project. 

These skills make you an irresistible MLOps candidate

  • Experience with container technologies (Docker, Kubernetes, EKS, ECS).
  • Experience with multiple cloud providers (AWS, GCP, Azure, etc).
  • Experience in distributed computing.

What’s next?

By now, you must’ve got a clear picture of what is an MLOps Engineer, how the role works, and what’s expected or required of them. However, the distinction between MLOps Engineers and ML Engineers, for example, is unclear. In a few words, the platforms built by MLOps professionals allow ML engineers and Data Scientists to work fast and effectively.

To those who question the fate of this role, the skills and responsibilities will grow for sure, but the job titles might change or get merged into some other similar titles.

That being said, it’s still a bright future. Regardless of what the future holds for MLOps engineers, best practices and new tools and workflows are here to stay and contribute even more to the challenges that the world of data presents. 

You can learn about these here:

MLOps Engineer FAQs

    • Familiarity with cloud platforms like AWS, Azure, or GCP, including knowledge of services like EC2, S3, SageMaker, or Google Cloud ML Engine for scalable and efficient model deployment.
    • Experience with Docker and container orchestration platforms like Kubernetes.
    • Experience with Quality Assurance using experiment tracking and workflow versioning.
    • Proficiency in data ingestion, pipelines, transformation, and storage technologies (e.g., SQL, NoSQL, Hadoop, Spark).
    • Experience with machine learning frameworks such as PyTorch, TensorFlow, and TFX.
    • Knowledge of version control (e.g., Git), CI/CD tools (e.g., Jenkins), and infrastructure automation (e.g., Ansible, Terraform)
    • Experience with unit/integration testing and monitoring tools (e.g., Prometheus, ELK Stack) and logging frameworks (e.g., Logstash, Fluentd)
    • May sound obvious, but a strong sense of teamwork and communication skills to collaborate with data scientists, engineers, and stakeholders will make you successful at your job.
  • What is the typical MLOps career path?

    • Fundamentals of ML – Gain knowledge of machine learning concepts, not too in-depth, but knowing what you’re deploying helps. Additionally, frameworks like TensorFlow, PyTorch are always part of the JD.
    • Software Engineering and Version Control – Get yourself well-versed in software engineering best practices and lifecycle, version control systems like Git, and debugging practices.
    • DevOps – Learn CI/CD, infrastructure automation, and cloud platforms like AWS, Azure, or GCP. Gain experience with tools like Jenkins, Docker, Kubernetes, and configuration management tools.
    • Gain Data Engineering Expertise – Learn Spark, NoSql, Hadoop to understand how to process large volumes of data.
    • MLOps tools – These will be your BAU, so, gain experience in Airflow, Kubeflow, AWS/Azure/GCP suit of MLOps tools (Sagemaker etc), DVC.
    • Communication skills – This might be a redundant thing, but an important part of MLOps jobs is to collaborate with different disciples.
  • Salary ranges from $100k to $170k and can vary depending on location, experience and industry. MLOps is a relatively new and evolving field, salary ranges can increase over time as the demand for skilled professionals increases.

    1. Have you implemented CI/CD pipelines for ML models? If so, explain the tools and practices you used.
    2. Describe a time when you had to handle data drift or concept drift in a deployed ML model. How did you address it?
    3. Describe your experience with containerization technologies like Docker and orchestration tools like Kubernetes in the context of ML deployments.
    4. Can you discuss any experience you have with feature engineering and feature stores in the context of ML workflows?
    5. What strategies would you use to monitor the performance of deployed ML models and handle anomalies or issues?

    Be wary that interview questions would largely revolve around concepts and tools that the company uses on-premise. Do some homework before walking into an interview. Good luck!

  • MLOps career path journey
    MLOps career path

(Bonus) Interviews with MLOps engineers

To learn more about the role of MLOps Engineer, we reached out to a few of them and asked a couple of questions. This might help you decide what’s in store for you.

To learn more about the role of MLOps Engineer, I reached out to a  few of them and asked a couple of questions. 

My guest professionals are:

  • Caroline Zago, Brazil, currently working at XP Inc. as MLOps Engineer;
  • Dmitry Goryunov, Germany, working as MLOps Engineer at deepset;
  • Maisa Daoud, New Zealand, working as MLOps Engineer at Servian;
  • Amy Bachir, United States, working as Senior MLOps Engineer at Interos Inc;
  • Alexey Grigorev, originally born in Russia but currently living in Berlin, Principal Data Scientist at OLX Group, founder of DataTalksClub, author of the Machine Learning Bookcamp, also recently taught the free Machine Learning Bootcamp;
  • and last but not least, Neal Lathia, UK, working as Associate Director of Machine Learning at Monzo Bank.

Each of these folks answered seven questions on the MLOps Engineer role that might help clear up some misconceptions.

How did you get started as an MLOps Engineer?

I have been in Software Engineering for more than ten years when I got interested in Machine Learning. There were a couple of courses on Coursera on the topic. They were not easy, but going through them was fun nevertheless. Later, I joined a Natural Language Processing team that had a few brilliant data scientists but no engineers.

From my Software Engineering background, I was used to the software development life cycle. You know, planning, development, testing, deployment, monitoring, etc. At first, I was horrified at how different it was for machine learning. It seemed like a wild west, where a data scientist trains a model in a Jupyter Notebook, measures its performance, and gives the result a binary file to an engineer.

The engineer deploys the model in the cloud, makes it part of the business logic, and leaves the binary file in production for good: no proper monitoring, no traceability. I am not even mentioning things like continuous integration. The worst part was that everybody just went to the next project after the binary was deployed.

I was lucky to have very understanding data scientists in my first team; together, we saw the MLOps role to ensure that the ML models benefit from the same best practices established in software development. Doing so assures that the ML models in production have more or less the same performance as on the test dataset.
 — Dmitry Goryunov MLOps Engineer at deepset
I was a DevOps engineer for a few years before I got into MLOps, so I already had experience in CI/CD (Continuous Integration/Continuous Deployment), GitOps, deployments, monitoring, and automation. However, I was missing the pieces that are unique and specific to the machine learning application lifecycle, so I started looking at online courses to learn the basics.

I found a very interesting and comprehensive nanodegree with Udacity for building machine learning models, so I took that nanodegree and graduated from it. After earning that nanodegree and having a solid experience in DevOps, it was easy to get my first role in MLOps
— Amy Bachir Senior MLOps Engineer at Interos Inc.
I started as a Data Scientist Intern, but I liked DevOps and Software Engineering. Then, I found MLOps, which brings together the three areas that I like. — Caroline Zago MLOps Engineer at XP Inc.

Is there any difference between ML Engineers and MLOps Engineers?  

Yes! Absolutely! In my opinion, ML Engineers build and retrain machine learning models. MLOps Engineers enable the ML Engineers. MLOps Engineers build and maintain a platform to enable the development and deployment of machine learning models. 

They typically do that through standardization, automation, and monitoring. MLOps Engineers reiterate the platform and processes to make the machine learning model development and deployment quicker, more reliable, reproducible, and efficient. — Amy Bachir Senior MLOps Engineer at Interos Inc.
Inside of Monzo, we do not (yet) have anyone who is called an “MLOps Engineer”. Instead, we have: Machine Learning Scientists who design, train, and ship models; Several teams of Backend Engineers who own and manage Monzo’s infrastructure, and a Backend Engineer who is focusing on ML-specific systems, like our feature store.
— Neal Lathia Associate Director of Machine Learning at Monzo Bank
For me, it is the same as the difference between engineering and Ops teams. Engineers create software, Ops provide the infrastructure for running it and make sure the software runs reliably. However, the lines are blurry, and MLOps Engineers can (and often should) do things end-to-end. — Alexey Grigorev Principal Data Scientist at OLX Group

What does MLOps Engineer do? At work, how is your daily routine?

We have a very early standup every morning, which is usually followed by a couple of separate meetings with colleagues to answer each other’s questions or set a plan to start working on a task. I work then till lunchtime, which I divide into 15 mins walk (treadmill) and 15 mins for a snack.

After that, I go back to the desk and have some little chats with colleagues about how things are going. If I finish a task at around 4 pm, then yeah, I feel free to look at some personal development course/reading as Servian leaders encourage us to stay updated. I usually finish at 6 pm in this case.
— Maisa Daoud MLOps Engineer at Servian
Oh, it depends. When you work at a company like Zalando, your work makes data scientists productive. So you work closely with data scientists to understand what they are about to do, to define their requirements. Then you try to fit their requirements inside the company’s infrastructure, build the data pipelines, deploy the servers to host the models, organize monitoring and continuous integration.

Now that I work for Deepset, I try to generalize my previous experience. Now it is more about building a convenient toolbox so that the data scientists who work with semantic search and question answering systems can train, evaluate, and deploy the question answering systems just by clicking a few buttons.
— Dmitry Goryunov MLOps Engineer at deepset
I start the day by making some coffee! I will then do things like joining planning and standup meetings, working with ML Scientists on specific work areas (by giving feedback on their designs and proposals), and working with other leaders across the company on areas of priority. If there are any problems with our systems, I will dive in to help resolve them. I am also spending a fair bit of time hiring and interviewing people right now. — Neal Lathia Associate Director of Machine Learning at Monzo Bank

What is the relationship between the MLOps team with the Data Science team at your company?

It is a very close relationship. We work together all the time. For the most part, everything we build is for them, so we consult with them before we design anything. We also run quarterly surveys to get their feedback on existing systems and what they think might be missing or is not working well for them. 

In general, I think it is very important to loop the end-users in everything we build because they are ultimately the consumers, and what we build should solve their problems and fit their needs. — Amy Bachir Senior MLOps Engineer at Interos Inc.
We do not have a separate data science team at OLX – we work in cross-functional packs where data scientists work together with other people on the same part of the product. However, we do have a central team that you may call MLOps. 

The role of the team is to help data scientists be more effective – especially in the teams that do not have a lot of engineering support. This team also standardizes some of the processes across all different packs. — Alexey Grigorev Principal Data Scientist at OLX Group

What do you think the future holds for this position?

I believe that this position will be increasingly requested in companies with a data science team because only a model running without the necessary structure for construction, production, and monitoring does not have the return expected by companies — Caroline Zago MLOps Engineer at XP Inc.
We see many tools emerging these days that automate MLOps. If we continue in this direction in a few years, data scientist teams might not need the support of a dedicated MLOps Engineer. This is what you work on in Neptune, and this is what we do in Deepset.

On the opposite side of this trend, we see many models available off the shelf and used by people who might not have a machine learning background. Products like Haystack, Hugging Face, and Neptune Model Registry democratize machine learning; they make it easier to make ML part of your application. 

These two trends might lead to the future where one specialist takes care of the complete life cycle of a model. — Dmitry Goryunov MLOps Engineer at deepset
As knowledge of machine learning becomes increasingly democratized, I believe that machine learning will increasingly intersect with Engineering and will not need to be its own separate thing. I expect that in the (near) future, it will become normal for Engineers to train and ship ML systems as part of their daily routine, without needing to go to specialized teams or via specialized tools. It will all become mainstream. — Neal Lathia Associate Director of Machine Learning at Monzo Bank

In what way do you think this new role’s existence has impacted the tech industry?

MLOps is cloud-specific practice. The time I started my career in the industry, I used to see ML Engineers doing their job on-premise and deploying their models in a cloud provider – although this has slowly shifted into relying more and more on cloud services. 

MLOps is taking ML practices in another way; we use cloud services every step from notebooks to deployment, although this is not mandatory, a recommended best practice. — Maisa Daoud MLOps Engineer at Servian
Whenever a new job title appears in the market, it is a great opportunity to bring more people into the tech world. It is also a great opportunity to start building communities of people solving the same kind of problem. 

However, when a job title is new, it also means that most companies do not quite know what they want or expect from that role or how to fit it into their company best. This was the case for Data scientists several years ago. In practice, that meant that the experience of being a Data Scientist could be very different from one company to the next. — Neal Lathia Associate Director of Machine Learning at Monzo Bank

And, finally: what piece of advice would you give to someone who is considering a career in MLOps Engineering?

The most impactful machine learning systems are the ones that can be safely and quickly launched and can impact your company’s product positively. So my advice is to always keep an eye on whether the tools you build enable that to happen. — Neal Lathia Associate Director of Machine Learning at Monzo Bank
I do enjoy working as an MLOps Engineer as much as I enjoyed working on ML research before. MLOps roles are teamwork, so you need to be ready for the collaboration idea, especially since there is no single right way to do the job. It is fun, believe me.

Cloud providers are growing very fast, especially in terms of developing ML SDKs (Software Development Kit). You will need a lot of patience and self-motivation to keep yourself on track. — Maisa Daoud MLOps Engineer at Servian
This is a tricky one! MLOps is in a very exciting role! There is much room for innovation! You will never get bored. At the same time, it is a very challenging role. The combination of skills required to succeed at this role is almost impossible to have in one person so prepare yourself for much learning. — Amy Bachir Senior MLOps Engineer at Interos Inc.
I would say that it is extremely important to understand the problem-solving process through modeling, understand the steps to get to production, and study software engineering concepts focused on machine learning, such as optimization, testing, and monitoring. I also believe that DevOps concepts and, of course, data science should be part of the knowledge of an MLOps Engineer. — Caroline Zago MLOps Engineer at XP Inc.
Do not listen to data scientists advising about MLOps.ㅤㅤㅤㅤㅤㅤ ㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤ — Alexey Grigorev Principal Data Scientist at OLX Group
It depends on the background the person has. If somebody comes from a software engineer job family, as I do, the advice would be to learn at least the basics of machine learning.

I am not speaking about getting a Ph.D. in the area, more like going through an online course or reading a couple of books. It is easy to find much material online these days. Courses of Andrew Ng is how I got started.

It will not be easy to wrap your head around machine learning concepts and get a basic understanding. At least, it was not easy for me. But, I assure you, it pays off. The basic understanding of ML lets you speak the same language as your data scientists, which is important for ML projects’ success.

While studying ML, you might find out that one of the topics is more interesting than the others. If this is the case, go and read a few papers or watch a few videos explaining them. Videos work even better sometimes. Go deeper into a specific topic. It is more fun to learn about machine learning that way. — Dmitry Goryunov MLOps Engineer at deepset

Was the article useful?

Thank you for your feedback!