Packaging ML Models: Web Frameworks and MLOps

In this article, we’ll see the what, why and how, of the top packaging tools – web-based frameworks and MLOps – for Data Science and ML projects. Data scientists and machine learning engineers need specific tools for building, deploying and monitoring these projects end-to-end. 

We’ll go through several tools in detail, along with their key components and features. No need for an introduction – let’s go!

Might be useful

Find the best MLOps tools for your use case MLOps Tools Landscape

Machine Learning development lifecycle

All ML projects we deal with have lots of iterative steps that need to be performed synchronously to achieve the best results. It becomes tedious for the data science team to manage these steps manually when both the ML models and data keep growing. 

This gave rise to the idea of operationalizing the entire machine learning development lifecycle. But what’s the best way to do this? Ideally, we need to divide the ML project work between key stakeholders and every team member needs to work in a highly collaborative manner for the project to be successful.

Stakeholders of Machine Learning and Data Science lifecycle

To get an idea of how to approach any machine learning project in an iterative and automated manner, we need to first understand all the stakeholders of the average machine learning and data science lifecycle. SMEs, data scientists, and data engineers are the most popular roles, but every stakeholder plays a significant role in the success of an ML project. Let’s see what different team members are usually responsible for.

Subject Matter Expert (SME) 

• Focuses on highly important business questions.
• Ensures model performance meets business needs/goals.
• Analyzes the business needs & objectives.

Data Analyst / Business Analyst

• Handles the data analysis and EDA.
• Assists with developing data features.
• Optimizes & builds data extractions for ML processes (ETL).

Data Engineer

• Analyzes and organizes the data.
• Builds the vital data pipelines.
• Collaborates with SMEs, data analysts, data scientists & data architects on projects.

Data Scientist

• Develops models to answer questions brought up by SMEs.
• Applies Predictive Analytics & Statistics.
• Reviews model results, accuracy & retrains models.

ML Engineer

• Tests models and delivers them into production to produce business value.
• Designs and develops Machine Learning apps according to customer/client requirements.
• Extends and enriches the existing frameworks and libraries used for Deep Learning and Machine Learning 

Software Engineer

• Develops APIs or applications that work with ML models.
• Verifies Machine Learning models perform correctly with various tools.
• Integrates ML Apps as Web apps using APIs.

Machine Learning Architect

• Optimizes the architecture for Machine Learning models as part of production deployment.
• Enables scaling of the models to be deployed in production.
• Collaborates with every member of the data science team and guides on ML projects.

DevOps Engineer

• Handles Continuous Integration & Continuous Deployment pipelines for the models in all the environments.
• Handles the security, integration, and performance of architecture that supports the models.

BI (Business Intelligence) Developer

• Develops, deploys & maintains interfaces like data visualization, query tools, and business dashboards.
• Transforms the business requirements into technical ones. Sets business requirements for BI tools.

Read more

Roles in ML Team and How They Collaborate With Each Other

Stakeholders are usually segregated across different groups and teams. With ML packing tools in action, the stakeholders can calculate and avoid risks for the company as a collaborative group. Moving models from development to production and scaling up are common challenges that accompany ML projects. They can cause an extended delivery timeframe for implementing and deploying models.

ML packing tools were created to optimize the data science and ML lifecycle. Thanks to these tools, the communication & implementation process can be streamlined. You can avoid a situation where there are suddenly too many stakeholders to keep track of who does what because tools can keep track of this for you. Keep in mind, these roles & responsibilities can differ across various firms.

Top 5 problems faced in Data Science and ML projects

In data science projects, we might encounter the following five challenges:

1. High expectations with all the hype around

• With so much hype around machine learning, the expectations are usually set too high. Data science and ML need to be explained in terms of their limitations, rather than just the potential benefits.

• Marketers and media often fail to address the full reality of the situation, so they paint a pretty picture of powerful technologies autonomously solving problems. But AI and ML are complex technologies that take time to implement and fully leverage for any company. They consume a lot of resources to deliver ROI. Stakeholders in an ML lifecycle need to manage expectations from the start.

2. Models don’t like rushed timelines

• Creating a good machine learning model is a lot of work and data scientists aren’t always capable of foreseeing the exact development and implementation time. Data science projects shouldn’t have stringent milestones & deadlines.

• The data science team might succeed in less or more time than predicted. Businesses need to show patience and continue to provide the team with the resources they need. 

3. Models should be future-ready devoid of maintenance

• After the data science team put in so much hard work into building and testing models, the question often arises whether their models have learned all that they ever need to. 
• Machine learning models need to be continuously trained and maintained to stay future-ready. Teams and businesses should ensure they incorporate the costs of doing so when they start an ML project.

4. Data collection consumes more than half of the time

• Data plays a vital role in any data science use case. 60% of the work of a data science team lies in collecting data. For beginners to experiment with machine learning, they can easily find various websites that host publicly available data.

• To implement real-world scenarios, you need to collect data through web-scraping, through APIs, or (for solving business problems) collect data from clients (in this case, data scientists or ML engineers need to coordinate and collaborate with subject matter experts to collect the data).

• Data, once collected, can have an unstructured format in the database. Knowledge of data engineering is necessary to format the data properly.

5. Model and application deployment

The data science lifecycle follows these 7 steps of building an application:

1) Data collection,

2) Data cleaning, 

3) Feature engineering,

4) Analyzing patterns,

5) Model training and optimization,

6) Validating the model,

7) Model testing, deployment, and maintenance.

Ah, deployment! Researchers and developers can often perform all the building steps, but sometimes lack the skills for deployment. Bringing their applications into production has become one of the biggest challenges due to lack of practice and dependency issues, low understanding of underlying models with business, and unstable models. This is where DevOps engineers, ML engineers, can come into the picture to deploy your models into production. 

Check also

How These 8 Companies Implement MLOps – In-Depth Guide
Best 8 Machine Learning Model Deployment Tools That You Need to Know

Generally, many developers collect data from websites and start training their model but in real-world scenarios, a source needs to be acquired for data collection that is dynamic. Offline learning or batch learning can’t be used for this type of variable data. The system is trained and then it’s launched into production, running without learning anymore. The data and models might drift as they could change very often.

Why do we need packaging tools for ML projects?

We need a packing tool, like a web-based framework or MLOps tool, to avoid or mitigate the challenges faced by the stakeholders of our project.

Building a machine learning solution as an end-to-end system that improves over time is the main barrier to delivering the business value in data science projects. Packing tools help with this.

Learn more

ML Model Packaging [The Ultimate Guide]

Challenges in ML projects

The big two challenges of ML projects are:

1. Taking ML models to production, only 47% of models are fully deployed! (Source: Gartner, 2019)

The gap between the number of models created and those that actually make it to production is a huge speed bump for the success of enterprise machine learning. Once organizations can get beyond that, organizations will be able to get much greater value from their machine learning investment.

2. The time to deploy Machine Learning and Deep Learning models is very high. (Source: Algorithmia, 2020)

A survey shows that taking ML & DL models into production deployment — where it eventually starts adding business value — takes from 8 to 90 days on average. What’s worse, up to 75% of ML projects never got beyond the experimental phase.

Here are several additional challenges that make it hard for projects to reach production:

• Scalability issues with data and models,
• Open-source pilots, not production grade,
• Difficulties in deployment into business applications and processes,
• Lack of DevOps and integration skills,
• Lack of funding & right tools,
• Data quality & integrity issues,
• Data governance & security for inputs and outputs.

Emerging roles in Data Science

Both traditional BI and trending AI are emerging, since both use data modeling (BI uses statistical techniques to analyze the past, and AI makes predictions).

Overall lack of resources in data science will result in an increasing number of software developers becoming involved in creating and managing Machine Learning models. (Gartner CIO survey)

More role names and job titles for the same type of work will emerge as a result. To that end, we’re seeing an influx of trending job titles in data science such as:

• Machine Learning Engineer, 
• ML Developer, 
• ML Architect, 
• Data Engineer, 
• Deep Learning Specialist, 
• NLP Engineer, 
• Computer Vision Engineer, 
• Machine Learning Operations (MLOps), 
• AI Ops.

The industry expands and firms attempt to distinguish themselves and their talent from the pack.

As a logical solution to the challenges and problems of taking machine learning models to production, MLOps is the new trend that has emerged alongside the existing way of packing frameworks for web and mobile development.

How to package ML projects

It’s important to package the ML project pipeline (or data science lifecycle, as in the image below) into an end-to-end system to achieve good ROI.

There are two methods to package DS and ML Projects:

1. Web based-frameworks
2. MLOps

1. Packaging ML projects with web-based frameworks

A web-based framework is a code library that makes web development faster, efficient, and easier by providing common patterns for building reliable, scalable, and maintainable web applications. Most of the time, professional web development projects always use an existing web framework like Flask, FastAPI, Django, or Pyramid.

Advantages of frameworks

1. Open-source,
2. Good documentation and community,
3. Efficient,
4. Secure,
5. Easy integrations.

Why are web-based frameworks useful?

Web-Based Frameworks make it easy for code reusability for HTTP operations that are very common & to structure projects such that anyone with the framework knowledge can swiftly build, maintain, and support the applications.

The functionality of common web-based frameworks

Frameworks provide functionality that performs common operations that include:

• Input form handling and validation – The idea is to validate the data and then save the form if it takes some input,
• URL Routing – The mechanism of routing is used for mapping the URL directly to the code that creates the web pages eventually.
• Output formats with template engine –  Generate any broadly used content types like HTML, XML, and JSON using a template engine.
• Database connection –  Persistent data manipulation and database connection configuration via Object Relational Mapping (ORM).
• Web security – Frameworks give web security against cross-site scripting (XSS), cross-site request forgery (CSRF), SQL Injection, and other malicious attacks.
• Session storage and retrieval – Anytime the page session ends, the data stored in the session storage gets cleared.

Full-stack framework

A full-stack framework is an all-in-one place with libraries configured to work seamlessly with one another. It can help you build back-end services, databases, and the front end. A full-stack framework provides anything a software developer needs to build an app.

Microframework

A microframework doesn’t have most of the functionality of a full-stack framework, such as a web template engine, accounts, authorization, authentication, or input validation. A microframework provides only the component set necessary for the app.

4 amazing Python web-based frameworks

1. Flask (microframework)

Flask is a famous and widely used Python web framework. It’s a third-party Python library used for developing web-based applications. Flask depends on the Werkzeug WSGI toolbox and Jinja2 template. The ultimate purpose is to help develop a strong web application base.

Key features of Flask:

• Built-in development server and A fast debugger.
• RESTful for dispatching of requests.
• Request handling for HTTP.
• Integrated unit testing support (code with quality).
• Plug & Play any Object Relational Mapping (ORM).
• Uses Jinja2 templating (tags, filters, macros, and more).
• Unicode-based.
• 100% WSGI 1.0 compliant.
• Multiple extensions provided by the community ease the integration of new functionalities.
• Secure cookies support.

2. FastAPI (microframework)

FastAPI is a modern web-based framework for API building with Python version 3.6+ based on standard Python types, which provides high-performance.

Key features of FastAPI:

• Very Good performance, close to NodeJS and Go, as it uses Starlette and Pydantic. Among the fastest Python frameworks present.
• Robust: Fetch production-ready code. With amazingly interactive documentation.
• Code speed: Improve the speed to develop feature components close to 300%*
• Standards-based: Fully compatible with open standards for APIs, OpenAPI, and JSON Schema.
• Lesser bugs: Decrease about 40% of developer-induced errors. *
• Learn easily to use & Less time-consuming docs.
• Short:  Multiple features from each parameter declaration.
• Intuitive: Great editor support, completion everywhere, less debugging time.

3. Django (full-stack framework)

Django is a Web-based Python framework that is high-level, which enables the development of secure and maintainable websites rapidly. Django removes most of the issues of web development so writing your applications becomes very easy, without having to reinvent the wheel.

Key features of Django:

• Ridiculously fast – Helps all developers take their web-based applications from concept to completion as swiftly as possible.
• Fully loaded – Looks after content administration, user authentication, site maps, RSS feeds, and many more tasks right out of the box.
• Exceedingly scalable – Django’s ability to swiftly and flexibly scale is a marvelous feature.
• Reassuringly secure – To avoid many common security mistakes, Django takes security seriously and helps developers.
• Incredibly versatile – Firms and governments have used Django to build all sorts of things from content management systems, through social networks, to scientific computing websites.
• Assists you to define patterns for URLs in your app.
• An authentication system that is eventually Built-in.
• Simple yet powerful URL system and routing.
• Cache framework, Template engine, Database Schema migrations, and ORM.

4. Pyramid (in-between full-stack and microframeworks)

Pyramid is an open-source and lightweight Python web framework focussed on taking small apps into big ones. Pyramid is based on the Model-View-Controller (MVC) architectural pattern, which is a WSGI web framework. Pyramid has the motto – Start Small, Finish Big, Stay Finished Framework built to make a bridge between full-stack and microframeworks.

Key features of Pyramid:

• All-embracing templating & asset details.
• Capability to run well with both small and large apps.
• The flexibility of both Authentication and approval.
• 2 different ways – WebHelpers & URL dispatch can be used for URL mapping based on route configuration.
• Testing, support & comprehensive data documentation.
• Validation & generation of HTML structure.

Differences between web-based frameworks

• Moving from Flask to FastAPI: https://testdriven.io/blog/moving-from-flask-to-fastapi
• Django vs Flask vs Pyramid: https://www.airpair.com/python/posts/django-flask-pyramid
• Django’s ORM cannot work without significant modification on non-relational (NoSQL) databases such as MongoDB or Cassandra whereas Django’s ORM layer allows a developer to write relational database read, write, query & delete operations in Python code instead of doing the same in SQL.
• Web frameworks, such as Pyramid & Flask, are easier to use with non-relational DB by using few Python libraries.

Now, onto the second method to package ML projects – MLOps.

2. Packaging ML projects with MLOps (Machine Learning Operations)

“ML Operations is at the intersection of three different but collaborative fields – Machine Learning, DevOps & Data Engineering, for operationalizing your AI models.“

MLOps is the discipline of Artificial Intelligence model delivery. MLOps empowers you and your firm to scale the capacity in a Production environment to deliver quicker and efficient results, generating significant value.

DataOps offers an amazing path to operationalize your data and AI platforms by extending the concepts of DevOps to the world of data. An extension of DevOps, DataOps is built on a rather simple framework of CI/CD: continuous integration, continuous delivery, and continuous deployment.

When Data Science and Machine Learning projects lack a solid framework & architecture to support model building, deployment, governance, & monitoring — they fail. To succeed, you need collaboration between data science team members like Data Scientists, DevOps Engineers, Data Analysts, Data Architects, and Data Engineers for productizing ML algorithms and delivering them to production at scale. When you extend this simple but effective framework further to the data marketplace, you get the solid framework that is MLOps.

4 core principles of MLOps

1. Continuous Integration (CI) – Continuous testing and validating of code, data, and models.
2. Continuous Delivery (CD) – Delivery of an ML training pipeline that automatically deploys another ML model prediction service.
3. Continuous Training (CT) – Automatically retraining ML models for redeployment.
4. Continuous Monitoring (CM) – Monitoring production data and model performance metrics.

Benefits of MLOps and why do we need it?

1. Accelerate time-to-value

Optimize the deployment of ML modes pragmatically to shorten the time from training to production—from months to minutes. So that it continues to improve & optimize to scale efficiently.

2. Optimize the productivity of teams

Integrate with existing frameworks, workflows & tools to provide transparent roles and optimize bottlenecks. Provide regular access to monitor and report on all the projects to make appropriate decisions.

3. Manage infrastructure

Systemically control the computational resources over models to adhere to business outcomes and cost-performance demands to dramatically lower prices. Deploy anywhere you want on-prem, cloud, or a hybrid environment.

4. Protect your business

Establish enterprise-grade role-based access and security controls across users, data, ML models, and resources to ensure the progression of your business through continuous delivery.

What problems does MLOps solve?

Data must always have a primary business focus. Operationalization helps to close the loop between gaining insight & turning that valuable insight into actionable value. Simple idea, yet the execution is anything but simple.

Establishing a Machine Learning Operations strategy could benefit your company or team in many ways:

• The roadblock which results from complex, black-box ML algorithms is reduced, with a better distribution of expertise and collaboration from operations & data teams.

• As machine learning has become mainstream, the regulatory front of operations is a critical function. ML Operations puts your operations teams at the forefront of new rules & best practices. They can take ownership of regulatory processes while your data team focuses on deploying amazing models.

• Your Ops team has the industry knowledge & your DS team understands the data. MLOps combines the expertise of both parties for effective ML that leverages both sets of experiences.

Comparison table of specific features of each MLOps tool

5 MLOps tools

1. MLflow

MLflow by Databricks is an open-source ML lifecycle management platform. It has tools to monitor your ML model during training and while running, along with the capability to store models, load models in production code, and build pipelines. MLflow is fit for individuals and for teams of all sizes. This amazing tool is library-agnostic. You can use it with any ML library and programming language you want.

Key Components:

• MLflow Tracking – Record and query experiments: data, config, code & results. For queries, It provides a web interface.
• MLflow Projects – Bundle ML code in a format to reproduce runs on any of the platforms. It always presents a format for Packing DS code in a way that is reusable and reproducible. In addition to this, API and command-line tools are included in the Projects component for running ML projects, making it desirable to chain together projects into workflows.
• MLflow Models – In several assisting environments, ML models can be deployed. For packing machine learning models, it uses a standard format that can be used in a mixture of downstream tools. For example, real-time serving through batch inference on Apache Spark or a REST API. The format defines a protocol that lets you save a machine learning model in diverse “flavors” that can be understood by various downstream tools.
• Model Registry – In a central repository, you can store, discover, manage & annotate ML models. 

Read also

The Best MLflow Alternatives
In-depth comparison between MLflow and Neptune

2. TensorFlow Lite & Tensorflow Extended

TensorFlow Lite is a deep learning-based framework applied for on-device inference. It’s an open-source framework that provides a set of tools to enable on-device ML and help developers run their models on mobile, IoT & embedded devices.

Key features of TensorFlow Lite:

1. On-device ML Optimized – handles five vital constraints: privacy, power consumption, connectivity, size, and latency.
2. Support for Multiple Platforms – Android & iOS devices, microcontrollers, embedded Linux.
3. Support for Diverse Languages – Python, Objective-C, Java, C++, and Swift.
4. Performance Made High – by the acceleration of hardware & optimization of models.
5. Illustrative Examples are End-to-End – Almost all common ML tasks such as question answering, object detection, pose estimation, image classification & text classification on various platforms.

TensorFlow Extended (TFX) is an end-to-end machine learning platform for deploying models and data pipelines in production, implemented by Google. It’s general-purpose and it evolved from TensorFlow. To move your models from research to production, you can use TFX to create and manage ML production pipelines with ease.

Key components:

• TensorFlow Data Validation – Explores & Validates data used for machine learning.
• TensorFlow Transformation – Creates transformation graphs that are consistently applied during training and serving to make full-pass analysis phases on the available data.
• TensorFlow Model Analysis – Computes full-pass & sliced model metrics on massive datasets, and analyzes them using libraries and visualization elements.
• TensorFlow Serving –  TF Serving is designed for production environments. It’s a flexible, high-performance serving system for machine learning models.

May interest you

️ Deep Dive into ML Models in Production Using TensorFlow Extended (TFX) and Kubeflow
️ The Best ML Frameworks & Extensions for TensorFlow

3. Azure ML and Azure Pipelines

Azure ML gives provision for the following MLOps capabilities :

• Helps Create ML pipelines that are reproducible. ML pipelines allow you to define re-usable & repeatable steps for your data preparation, training & model evaluation techniques to be easy.
• Creates software environments that are reusable for training & deploying Machine Learning models.
• From anywhere you can register, package, and deploy models. You can also follow the associated metadata needed to use the ML model.
• Governance data can be captured for the end-to-end ML lifecycle. The information that is logged includes why changes were made, who is publishing models & finally when the ML models were used in production.
• Notifies on completion of events in the Machine Learning lifecycle. Example: model registration, data drift detection, model deployment & experiment completion.
• Machine Learning apps can be monitored for ML-related & operational issues, like by monitoring to provide alerts on your ML infrastructure & metrics.
• Data Science and Machine Learning lifecycle automated using tools like Azure ML & Azure Pipelines. Employing pipelines helps you to regularly update Machine Learning models, test new models, and continuously roll out new models alongside your other apps & services.

4. Kubeflow

Kubeflow is the ultimate ML platform designed to enable using ML pipelines to orchestrate complex workflows running on Kubernetes. Kubeflow was initially based on Google’s internal way to deploy TensorFlow models called (TFX)TensorFlow Extended.

The Kubeflow project is dedicated to making deployments of workflows of Machine Learning on Kubernetes simple, scalable & portable. Their goal was to provide a straightforward way to deploy best-of-breed systems that are open-source for Machine Learning to diverse infrastructures.

Key features of Kubeflow:

• To manage and track experiments, runs, and jobs, Kubeflow provides a good user interface.
• Multi-framework integration & SDK to interact with the system using Notebooks.
• To swiftly build end-to-end solutions you can re-use all the provided components & pipelines.
• Pipelines in Kubeflow are either a core component of Kubeflow or a standalone installation.
• Perfect fit for every Kubernetes user. Wherever you’re running Kubernetes, you can run Kubeflow.

Read also

The Best Kubeflow Alternatives
In-depth comparison between Kubeflow and Neptune

5. AWS Sagemaker

Amazon SageMaker is a complete ecosystem for MLOps that enables data science teams to rapidly build, train, & deploy ML and DL models at any scale. AWS SageMaker incorporates modules that are used collectively or independently to train, build & deploy your AI & ML models. It has a Studio environment that blends Jupyter notebooks with experiment tracking & management, batch transforms, a model monitor, deployment with elastic inference, an “autopilot” for Auto ML for naive users, and a model debugger.

Key Highlights of AWS Sagemaker:

• Amazon Web Services (AWS) is a Visionary in this 2021 Gartner – Magic Quadrant. Most of the upholding AWS components & services were carefully analyzed in assessing AWS offerings. These included the Amazon EMR (including S3), AWS Glue, AWS CloudWatch, SageMaker Ground Truth, Amazon Clarify, Data Wrangler, SageMaker Studio IDE, SageMaker Pipelines, AWS CloudTrail, SageMaker Neo & many others.

• AWS is diversified based on geography, and its client base traverses across several industries & business functions.
• Amazon SageMaker remains a formidable player in the market traction, with considerable resources and a powerful ecosystem behind it.

Read also

In-depth comparison between SageMaker and Neptune.

Experiment tracking – how to make model packaging smoother

Keeping track of your Machine Learning experiments in a central place (using tools like Neptune) helps a lot. Neptune is a metadata repository for MLOps, developed for research and production teams to run a huge number of experiments (in millions). It gives a primary place to log, store, organize,  display, query, and compare all the metadata generated throughout the ML lifecycle.

Neptune Metadata Store for MLOps = Client library + Metadata Database + Dashboard/UI  (to filter, sort, and compare experiments).

Neptune gives an open-source Python library that permits users to log any experiments. The tool is intended to give an easy-to-use and quick-to-learn method to maintain the track of all experiments. It also comes with close to 30 integrations with Python libraries popular for ML, DL, and RL. 

3 major elements present in Neptune:

1. Data versioning – with helpers data location and data hash can be logged to Neptune.
2. Experiment tracking – compare experiments and models (optimization, classic ML, deep learning, reinforcement learning,) with absolutely no additional efforts.
3. Model registry – track model lineage & version.

These 3 major elements enable Neptune to serve as a bridge between several parts of the MLOps life cycle. 

Learn more

Dive deeper into Neptune’s features here.

Conclusion

And that’s it – the top 10 ML packing tools to pick up and use right now in your projects. Make sure to look for components, features, and relevant factors when selecting the platform that suits your specific needs. This way you’ll get the most out of your work.

Developing and deploying machine learning models into production has always been a vital element of the data science and ML lifecycle. In the past, it required a lot of effort from all of the ML lifecycle stakeholders, as we saw at the start of this article. Packing tools were limited and the process was manual and highly time-consuming. It’s not the case anymore thanks to these packing tools.

Hope this article helped you decide on the right ML packing tool for your project, one that will make your work more enjoyable and productive. Happy exploring and learning!

References

• https://steelkiwi.com/blog/top-10-python-web-frameworks-to-learn/
• https://acuvate.com/blog/challenges-data-scientists-face-in-machine-learning/