MLOps is not a piece of cake. Especially in today’s changing environment. There are many challenges—construction, integrating, testing, releasing, deployment, and infrastructure management. You need to follow good practices and know how to adjust to the challenges.
And if you don’t learn and develop your knowledge, you’ll fall out of the loop. The right resources can help you follow the best practices, discover helpful tips, and learn about the latest trends.
You don’t have to look far, we’ve got you covered! Here’s your list of the best go-to resources about MLOps—books, articles, podcasts, and more. Let’s dive in!
1. Introducing MLOps from O’Reilly
Introducing MLOps: How to Scale Machine Learning in the Enterprise is a book written by Mark Treveil and the Dataiku Team (collective authors). It introduces the key concepts of MLOps, shows how to maintain and improve ML models over time, and tackles the challenges of MLOps.
The book was written specifically for analytics and IT operations team managers—the people directly facing the task of scaling machine learning (ML) in production. It’s a guide for creating a successful MLOps environment, from the organizational to the technical challenges involved.
The book is divided into three parts:
- An introduction to the topic of MLOps, how and why it has developed as a discipline, who needs to be involved to execute MLOps successfully, and what components are required.
- The second part follows the machine learning model life cycle, with chapters on developing models, preparing for production, deploying to production, monitoring, and governance.
- Provides tangible examples of how MLOps looks in companies today, so readers can understand the setup and implications in practice.
2. What Is MLOps? from O’Reilly
What Is MLOps? Generating Long-Term Value from Data Science & Machine Learning by Mark Treveil and Lynn Heidmann is a thorough report for business leaders who want to understand and learn about MLOps as a process for generating long-term value while reducing the risk associated with data science, ML, and AI projects.
Here’s what the report includes:
- Detailed components of ML model building, including how business insights can provide value to the technical team
- Monitoring and iteration steps in the AI project lifecycle–and the role business plays in both processes
- How components of a modern AI governance strategy are intertwined with MLOps
- Guidelines for aligning people, defining processes, and assembling the technology necessary to get started with MLOps.
This is An awesome list of references for MLOps – Machine Learning Operations from ml-ops.org
It’s a list of links to numerous resources, beginning with books, articles, to communities, and many, many more. In a word—it has everything you could possibly read about MLOps. The table of contents includes among others: MLOps Papers, Talks About MLOps, Existing ML Systems, Machine Learning, Software Engineering Product Management for ML/AI, The Economics of ML/AI, Model Governance, Ethics, Responsible AI.
Be careful! It might be a tough and long read if you want to go through all the links, but if you want to learn all about MLOps, it’s one of the best resources. 😉
👉 Additionally, you can check the official website of MLOps for more interesting information and resources to expand your knowledge and learn about the best practices.
4. Practical MLOps from O’Reilly
The book Practical MLOps: Operationalizing Machine Learning Models by Noah gift and Alfredo Deza is an insightful guide that takes you through what MLOps is, how it differs from DevOps, and shows you how to put it into practice to operationalize your machine learning models.
This is what you’ll learn from the book:
- Apply DevOps best practices to machine learning
- Build production machine learning systems and maintain them
- Monitor, instrument, load-test, and operationalize machine learning systems
- Choose the correct MLOps tools for a given machine learning task
- Run machine learning models on a variety of platforms and devices, including mobile phones and specialized hardware.
👉 Here’s the free version on GitHub. It’s a public repo where code samples are stored for the book Practical MLOps.
The Stanford MLSys Seminar Series is, as the name suggests, a series of seminars focused on machine learning and ML systems—tools and all the technology used for programming machine learning models.
The course started in fall 2020. Every talk is live-streamed in this seminar series Thursdays 1-2 PT on YouTube. You can ask questions on the live chat. Videos of the talks are available on YouTube afterward as well. Give the channel a follow here, and tune in every week for an exciting discussion!
The goal of the course is to help curate a curriculum of awesome work in ML systems to help drive research focus to interesting questions.
6. MLOps.community Podcast
MLOps.community is a podcast hosted by Demetrios Brinkmann. It has “weekly talks and fireside chats about everything that has to do with the new space emerging around DevOps for Machine Learning aka MLOps aka Machine Learning Operations.”
There are interviews, conversations with interesting people, tips, talks about challenges, trends, and more. Tune in and listen!
You can learn many interesting things and broaden your knowledge when talking to other people and exchanging your knowledge. This Slack channel is an open community for all enthusiasts of ML and MLOps, be it amateurs or professionals.
You can meet every Wednesday at 5pm UK time on Zoom to listen to people giving interesting talks. And if you’d like to, you can register yourself to share your knowledge with the rest of the world!
8. Google Cloud
MLOps: Continuous delivery and automation pipelines in machine learning is a document from Google that “discusses techniques for implementing and automating continuous integration (CI), continuous delivery (CD), and continuous training (CT) for machine learning (ML) systems.”
If you’re new to MLOps, this document can be a great source of knowledge as it touches on some basic concepts. But if you’re the MLOps veteran, you’ll also find it helpful to refresh and solidify your knowledge. It can also help reliably build and operate ML systems at scale.
To wrap it up
MLOps is important if you want to build high-quality models for your ML experiments. It can boost your production pipeline and improve the team’s performance. So keep learning to implement the best practices and always stay on top!
And don’t forget to follow our blog for the latest articles. We’re publishing regularly to bring you the best content. We also have a section dedicated to MLOps, you can find it here.
Are we missing something? Let us know if you’re not seeing your favorite resources on our list!
ML Experiment Tracking: What It Is, Why It Matters, and How to Implement It
Jakub Czakon | Posted November 26, 2020
Let me share a story that I’ve heard too many times.
”… We were developing an ML model with my team, we ran a lot of experiments and got promising results…
…unfortunately, we couldn’t tell exactly what performed best because we forgot to save some model parameters and dataset versions…
…after a few weeks, we weren’t even sure what we have actually tried and we needed to re-run pretty much everything”
– unfortunate ML researcher.
And the truth is, when you develop ML models you will run a lot of experiments.
Those experiments may:
- use different models and model hyperparameters
- use different training or evaluation data,
- run different code (including this small change that you wanted to test quickly)
- run the same code in a different environment (not knowing which PyTorch or Tensorflow version was installed)
And as a result, they can produce completely different evaluation metrics.
Keeping track of all that information can very quickly become really hard. Especially if you want to organize and compare those experiments and feel confident that you know which setup produced the best result.
This is where ML experiment tracking comes in.Continue reading ->