There is a broad catalog of tools you can use as helping hands to increase your understanding of machine learning models. They come in different categories:
- Interactive web app tools
- Data analysis tools
- Model explainability tools
- Model debugging tools
- Model performance debugging tools
- Experiment tracking tools
- Production monitoring tools
In this article, I’ll briefly tell you about as many tools as I can, to show you how rich the ML tools ecosystem is.
1. Interactive web app tools
This open-source ML tool lets you build customized web apps for your models. You can showcase your model in a very interactive and understandable way so that anybody can use it easily.
In just a few minutes, you can build and deploy beautiful and powerful data apps.
To use Streamlit, you just need to install it with pip by using the command:
pip install streamlit
Streamlit’s UI will ask if you’d like to rerun the app and view the changes. This allows you to work in a fast iterative loop: you write some code, save it, review the output, write some more, and so on until you’re happy with the results.
Check out the Streamlit docs to see exactly how it works.
One of the most popular, lightweight Python frameworks for building web apps. You can use it to develop a web API for your model. It’s based on the Werkzeug WSGI toolkit and Jinja2 template engine.
Flask has a simple architecture, and you can learn it very easily. It’s highly recommended for building small applications. You can quickly deploy the model and set up a REST API.
To start using Flask, set up the virtual environment, and install it using the command:
pip install flask
For detailed information, check out Flask documentation.
Shiny is an R package for building interactive web apps. If you already know the R language, it will be really easy to use to build an app and share your work. It has a really interactive and intuitive design.
To start using Shiny use the command:
2. Data analysis tools
DABL stands for Data Analysis Baseline Library. You can use it to automate repetitive processes that happen in the early stages of model development, like data-cleaning, preprocessing, analysis, dealing with missing values, or converting data to different formats.
It’s a new Python library, so its functions are limited, but it’s very promising.
To start using DABL use the command:
pip install dabl
For more information about DABL, check out this article.
Open-source data analysis tool for creating data science applications and building machine learning models.
You can integrate various components for machine learning and data mining through its modular data pipelining concept. KNIME has been used in areas like CRM, text mining, and business intelligence.
It provides an interactive GUI to create workflows using a drag and drop builder. It supports multi-threaded in-memory data processing, and KNIME server supports team-based collaboration.
To get started with KNIME, visit the documentation.
Data science platform that helps you prepare and analyze data. Very user-friendly, you can just drag and drop pieces of code.
It has several data exploration features that you can use to gain valuable insights from your data. It provides more than 14,000 operators for data analysis.
To get started with RapidMiner, follow this link.
SAS stands for Statistical Analysis System, and it’s used to analyze statistical data. It helps you do data analysis using SAS SQL and automatic code generation. You can easily integrate it with MS tools like Excel.
SAS lets you create interactive dashboards and reports to better understand complex data.
To get started with SAS, check out this tutorial.
3. Model Explainability tools
Eli5 is a Python package that lets you explain predictions of machine learning classifiers. It provides support for the following packages and frameworks:
- XGBoost – explains predictions of XGBClassifier, XGBRegressor, and also helps validate feature importance.
- CatBoost – explains predictions of CatBoostClassifier, CatBoostRegressor, and also helps validate feature importance.
- Scikit-learn – explains weights and predictions of scikit-learn linear classifiers and regressors, and also helps validate feature importances of decision trees.
To learn more about Eli5, check out the documentation.
It stands for SHapley Additive exPlanations. It’s based on Shapley values. SHAP values show the impact of each feature by breaking down the predictions, which may result in powerful model insights.
Some applications of SHAP values are:
- A model says a bank shouldn’t loan someone money, and the bank is legally required to explain the basis for each loan rejection.
- A healthcare provider wants to identify what factors are driving a patient’s risk of a disease, in order to directly address those risk factors with targeted health interventions.
To learn more about SHAP, check out this tutorial.
It stands for Descriptive Machine Learning Explanations. It’s an R package, built mainly for model interpretability.
Machine learning interpretability is becoming more important. This package helps you extract insights and get clarity on how your algorithm works, and why one prediction is made over another.
Dalex makes it convenient to compare performance across multiple models. There are several advantages of using Dalex:
- It includes a unique and intuitive approach for local interpretation,
- It can customize predicted outputs,
- It provides convenient ways for result comparisons.
To learn more about Dalex, check out the official GitHub page.
Neptune’s integration with Dalex 👇
4. Model debugging tools
A visual debugging tool for machine learning, built by Uber’s team. It was developed to make the model iteration process more informed and actionable.
Data scientists can use it to look at a general summary and detect the inaccurately predicting part of data. Manifold also explains the potential causes of poor model performance by surfacing the feature distribution differences between better and worse-performing subsets of data.
Manifold serves the majority of the ML models, which includes most classification and regression models.
To learn more about Uber Manifold, check out the official page.
5. Model performance debugging tools
MLPerf is becoming a staple in ML workloads for interesting experiments, comparing different types of specialized infrastructures or software frameworks. It’s used for building useful standards for measuring training performance of ML hardware, software, and services.
The goals of MLPerf are to serve both commercial and academic communities, to ensure reliable results, to accelerate progress in ML, and to enable fair comparison of competing systems, yet to encourage innovation to improve state-of-the-art ML.
To learn more about MLPerf, check out this introductory article.
6. Experiment tracking tools
It stands for Weights and Biases. It’s a Python package that lets you monitor model training in real-time. It easily integrates with popular frameworks like Pytorch, Keras, and Tensorflow.
Additionally, it lets you organize Runs into Projects, where you can easily compare them and identify the best performing model.
Learn more about WandB, in this introductory article.
Check the comparison between WandB and Neptune.ai.
Lightweight and powerful experiment tracking tool for data scientists. It easily integrates with your workflow and offers an extensive range of tracking features.
You can use it to track, retrieve, and analyze experiments, or to share them with your team and managers. Neptune is flexible, works with many frameworks, and thanks to its stable user interface, it enables great scalability (to millions of runs).
It also lets you store, retrieve, and analyze large amounts of data.
To learn more about Neptune, check the website.
Comet helps data scientists manage and organize machine learning experiments. It lets you easily compare experiments and keep a record of collected data, as well as collaborate with other team members. It can easily adapt to any machine and works well with different ML libraries.
To learn more about Comet, check out the official documentation.
Check the comparison between Comet and Neptune.ai.
Open-source platform for tracking machine learning experiments, and also for deploying models. Each element is represented by one MLflow component: Tracking, Projects, and Models.
This means that if you’re working with MLflow, you can easily track an experiment, organize it, describe it for other ML engineers, and pack it into a machine learning model.
MLflow has been designed to enable scalability from one person to a big organization, but it works best for an individual user.
To learn more about MLflow, check the official documentation.
– Check which tool is better: Neptune vs MLflow
– Explore other tools in the article 15 Best Tools for Tracking Machine Learning Experiments
7. Production monitoring tools
Kubeflow makes it easier to deploy machine learning workflows. It’s known as the machine learning toolkit for Kubernetes and aims to use the Kubernetes potential to facilitate scaling of ML models.
The team behind Kubeflow is constantly developing its features, and doing its best to make life easier for data scientists. Kubeflow has some tracking capabilities, but they’re not the main focus of the project. It can be easily used with other tools on this list as a complementary tool.
To learn more about Kubeflow, check out the official documentation.
Check the comparison between Kubeflow and Neptune.ai.
This concludes our list of different machine learning tools. As you can see, the ecosystem is wide, and the list doesn’t cover all the tools out there.
Whatever are your needs, you don’t need to do things manually. Use these tools to speed up your workflows, and make your life as a data scientist easier.
Get started with Neptune in 5 minutes
If you are looking for an experiment tracking tool you may want to take a look at Neptune.
It takes literally 5 minutes to set up and as one of our happy users said:
“Within the first few tens of runs, I realized how complete the tracking was – not just one or two numbers, but also the exact state of the code, the best-quality model snapshot stored to the cloud, the ability to quickly add notes on a particular experiment. My old methods were such a mess by comparison.” – Edward Dixon, Data Scientist @intel
To get started follow these 4 simple steps.
Install the client library.
pip install neptune-client
Connect to the tool by adding a snippet to your training code.
import neptune neptune.init(...) # credentials neptune.create_experiment() # start logger
Specify what you want to log:
neptune.log_metric('accuracy', 0.92) for prediction_image in worst_predictions: neptune.log_image('worst predictions', prediction_image)
Run your experiment as you normally would:
And that’s it!
Your experiment is logged to a central experiment database and displayed in the experiment dashboard, where you can search, compare, and drill down to whatever information you need.