Metadata store for MLOps, built for research and production teams that run a lot of experiments

Feel in control of your models and experiments by having all metadata organized in a single place.
Focus on ML, leave metadata bookkeeping to Neptune.
Get started in 5 minutes.

What is a metadata store for MLOps?

ML metadata store is an essential part of the MLOps stack that deals with model building metadata management.
It makes it easy to log, store, display, organize, compare and query all metadata generated during ML model lifecycle. 

What ML metadata are we talking about?

Experiment and model training metadata

You can log anything that happens during ML run including:

  • Metrics
  • Hyperparameters
  • Learning curves
  • Training code and configuration files
  • Predictions (images, tables, etc)
  • Diagnostic charts (Confusion matrix, ROC curve, etc)
  • Console logs
  • Hardware logs
  • And more
See what else you can log
Learning curves

Artifact metadata

For datasets, predictions or models you can log:

  • Paths to the dataset or model (s3 bucket, filesystem)
  • Dataset hash
  • Dataset/prediction preview (head of the table, snapshot of the image folder)
  • Description
  • Feature column names (for tabular data)
  • Who created/modified 
  • When last modified
  • Size of the dataset
  • And more
See what else you can log
Artifact metadata

Model metadata

For trained models (production or not) you can log:

  • Model binary or location to your model asset
  • Dataset versions 
  • Links to recorded model training runs and experiments 
  • Who trained the model
  • Model descriptions and notes
  • Links to observability dashboards (Grafana)
  • And more
See what else you can log
Model metadata

What can you use an ML metadata store for?

Experiment tracking

Track, organize, and compare everything you care about in your ML experiments.

  • Monitor experiments as they are running
  • Keep track of metrics, parameters, diagnostic charts, and more 
  • Search, group, and compare experiments with no effort
  • Drill down to every experiment information you need
  • Share results with your team and access all experiment details programmatically
Learn more
Experiment_tracking

Model registry

Have your models versioned, traceable and easily accessible. 

  • Know exactly how every model was built
  • Record dataset, code, parameters, model binaries, and more for every training run
  • Get back to every model building metadata even months after 
  • Share models with your team and access them programmatically
Learn more
Source code

Neptune Metadata Store for MLOps = Client + Database + Dashboard

Metadata_dashboard

Client library

It makes it easy to log and query the ML metadata database.

Check the docs
# log
experiment.log_metric('accuracy', 0.9)
experiment.log_image('predictions', image_pred)
experiment.log_artifact('model_weights.h5')
# query
project.get_experiment(id='EXP-123').download_artifacts()

Metadata Database

It’s a place where experiment, model, and dataset metadata are stored. They can be logged and queried efficiently.

See example dashboard
Metadata database

Dashboard

A visual interface to the metadata database. A place where you can see metadata of your experiment, models, and datasets.

See example project

Get started in 5 minutes

1. Create a free account
Sign up
2. Install Neptune client library
pip install neptune-client
3. Add logging to your script
import neptune

neptune.init('Me/MyProject')
neptune.create_experiment(params={'lr':0.1, 'dropout':0.4})
# training and evaluation logic
neptune.log_metric('test_accuracy', 0.84)
Try live notebook

Dive deeper into Neptune features

Log and display any ML metadata you care about

How can you start logging?

To log things to the Neptune experiment, you need
an experiment 🙂. 

  • Connect Neptune to your script (pass credentials)
  • Create an experiment (start logging context)
  • Log whatever you want
Try live notebook
neptune.init('YOUR_ORG/PROJECT', 'YOUR_API_TOKEN') # Credentials
neptune.create_experiment('great-idea') # Start logging context
neptune.log_metric('accuracy', 0.92) # Log things

What can you log and display?

Neptune allows you to log most metadata related to ML models, experiments, and datasets, including:

  • Training code and configuration files
  • Parameters and metrics
  • Hardware consumption and console logs
  • Performance charts and interactive visualizations
  • Model weights 
  • and more!
Check the docs
Logging metadata

Log more with less code with our integrations!

Don’t implement the loggers yourself. We have 25 + integrations with all major ML frameworks to make things even easier. 

  • PyTorch and PyTorch Lightning
  • TensorFlow / Keras and TensorBoard
  • Scikit-Learn, LightGBM, and XGBoost
  • Optuna, Scikit Optiimize, and Keras Tuner
  • Bokeh, Altair, Plotly, and Matplotlib
  • and more!
See integrations
#Tensorflow/Keras
model.fit(..., callbacks=[NeptuneMonitor()])

# PyTorch lightning
trainer = pl.Trainer(..., logger=NeptuneLogger())

# lightGBM
gbm = lgb.train(... callbacks = [neptune_monitor()])

# XGBoost
xgb.train(..., callbacks=[neptune_callback()])

# Optuna
study.optimize(…, callbacks=[NeptuneCallback()])

Compare experiments and models with no extra effort

Experiment auto-diff in a table

Compare metrics and parameters in a table that automatically finds what changed between experiments.

See the difference between metrics, parameters, text, and more with no extra effort!

Check the docs
Compare experiments table

Compare learning curves

Overlay learning curves and drill down to experiment details like interactive performance charts to get a better picture.

Get all that automatically just because you logged metadata to Neptune. 

Check the docs
Compare experiments

Organize experiments and model training runs in a single place

Run experiments everywhere, keep the results in one place

You can execute experiment code on your laptop, cloud environment, or a cluster. All ML metadata you care about will be logged to the central storage hosted by us or deployed on-premises. 

It works out-of-the-box with Python, R, Jupyter notebooks, and other languages and environments.

See example dashboard
Organize-experiments

Structure your teams’ ML work in workspaces and projects

Clean up your ML teamwork by grouping your experiment into projects and workspaces.

If you are working on a model that does a particular feature, just create a project for it. All the experiments in that project will be easy to compare and search through.

If your team is working for multiple clients or departments, you can create a workspace for each client and separate projects in each organization.

See example project
Team workspace

Filter, sort, and compare experiments in a dashboard

Search through your experiments quickly with a dashboard built for ML experiments and models.  

  • Filter experiments by metric and parameter values
  • Display min/max/last value of a metric or loss series like validation loss
  • Compare everything with no extra effort
See example dashboard
Change dashboard

See (only) the information you want: customize and save dashboard views

Choose which metrics, parameters, or other information you want to see and customize your dashboard. 

Create multiple dashboard views and save them for later.

See example project
Save new dashboard

See your ML experiments live as they are running

See learning curves live

Take a look at the learning curves and training metrics of your models and compare them to past runs as they are running. 

Detect unpromising training runs and react quickly. 

Check the docs
Product_monitor learning curves

See hardware consumption whenever you want

You can monitor the hardware for your experiment runs automatically. See how much GPU/CPU and memory your model training runs are using.  

See performance problems quickly, even for multi-gpu training.

Check the docs
Hardware usage

Look at model predictions, console logs, and anything else during training

You can log model predictions after every epoch or console logs from a remote machine and have full control over your training process. 

Check the docs
Model predictions

Make experiments and model training runs reproducible and traceable

Every time you train your model, you can automatically record:

  • Code and environment versions
  • Data Versions
  • Model parameters
  • Model weights and other binaries 
  • Evaluation metrics, 
  • Model explanation charts,
  • Who trained the model,
  • and anything else you need

You will not forget to commit your changes because it happens automatically. 

See example project
neptune.create_experiment(
     params={'lr':0.21, 
             'data_version': get_md5('data/train.csv')},
     upload_source_files=['**.*.py',
                          'requirements.yaml')
neptune.log_metric('acc', 0.92)
neptune.log_image('explanations', explanation_fig)
neptune.log_artifact('model.pkl')

Share any result or visualizations with your team by sending a link

Want to discuss what you see right now in the application? 

Just share a link. It’s that simple. 

Neptune has a system of persistent links that makes sharing experiment details, comparisons, dashboard views, or anything else straightforward.

See example project
Share with a link

Query experiment and model training metadata programmatically

You can download everything you logged to Neptune programmatically (or from the UI). 

  • Experiment table with all metrics and parameters
  • Experiment object itself and access all the metadata logged to it
  • You can update experiment objects with new information even after training!
Check the docs
project = neptune.init('Project')

project.get_leaderboard()

exp = project.get_experiments(id='Proj-123')[0]
exp.get_parameters()
exp.download_artifact('model.pkl')
exp.download_sources()

Neptune integrates with your favourite frameworks and tools

With Neptune you can get more out of tools you use
every day. Neptune comes with 25+ integrations with
libraries used in machine learning, deep learning and
reinforcement learning.

Learn more

See what our users are saying

“Neptune allows us to keep all of our experiments organized in a single space. Being able to see my team’s work results any time I need makes it effortless to track progress and enables easier coordination.”

Michael Ulin
VP, Machine Learning @Zesty.ai

“For me the most important thing about Neptune is its flexibility. Even if I’m training with Keras or Tensorflow on my local laptop, and my colleagues are using fast.ai on a virtual machine, we can share our results in a common environment.”

Víctor Peinado
Senior NLP/ML Engineer @reply.ai

“Within the first few tens of runs, I realized how complete the tracking was – not just one or two numbers, but also the exact state of the code, the best-quality model snapshot stored to the cloud, the ability to quickly add notes on a particular experiment. My old methods were such a mess by comparison.”

Edward Dixon
Enterprise Data Scientist @Intel

“Neptune is making it easy to share results with my teammates. I’m sending them a link and telling what to look at, or I’m building a View on the experiments dashboard. I don’t need to generate it by myself, and everyone in my team have access to it.”

Ronert Obst
Head of Data Science @New Yorker

“Neptune is making it easy to share results with my teammates. I’m sending them a link and telling what to look at, or I’m building a View on the experiments dashboard. I don’t need to generate it by myself, and everyone in my team have access to it.”

Maciej Bartczak
Resarch Lead @Banacha Street

“Without information, I have in the Monitoring section I wouldn’t know that my experiments are running 10 times slower than they could. All of my experiments are being trained on separate machines which I can access only via ssh. If I would need to download and check all of this separately I would be rather discouraged. When I want to share my results I’m simply sending a link.”

Michał Kardas
Machine Learning Researcher @TensorCell

“Previously used tensorboard and azureml but Neptune is hugely better. In particular, getting started is really easy; documentation is excellent, and the layout of charts and parameters is much clearer.”

Simon Mackenzie
AI Engineer and Data Scientist
Load more

They already use Neptune to manage their ML metadata.
When will you?

Get started in 5 minutes.

Not convinced?

Try in a live Notebook (zero setup, no registration)

Try now

Explore example project

Example project
Go to project

Watch screencasts

Screencasts featured
Watch now

See the docs

Docs view
Check now