Compare

Neptune vs Kubeflow

neptune-logo
vs
Kubeflow

Commercial Requirements

Commercial requirements chevron
Standalone component or a part of a broader ML platform?

Standalone component. ML metadata store that focuses on experiment tracking and model registry

Part of the Kubernetes environment

Is the product available on-premises and / or in your private/public cloud?

Almost all popular cloud providers maintain their own distribution of Kubeflow. It can also be installed on-premises manually. Read about the differejnt installation options available here

Is the product delivered as commercial software, open-source software, or a managed cloud service?

Managed cloud service

The base product is open source, with managed distributions made available by cloud providers

What is the pricing model?

The open source version is free. Maintainers have their own priving plans for their managed distributions

SLOs / SLAs: Does the vendor provide guarantees around service levels?

Community support for the open source version. Different options available for managed distributions

Support: Does the vendor provide 24×7 support?

Not for the open source version. Different options available for managed distributions

SSO, ACL: does the vendor provide user access management?

Different options available for managed distributions. Read more here

Security policy and compliance

Not for the open source version. Different options available for managed distributions

General Capabilities

Setup chevron
What are the infrastructure requirements?

No special requirements other than having the neptune-client installed and access to the internet if using managed hosting. Check here for infrastructure requirements for on-prem deployment

Advanced setup required for manual install. Installation requirements for the managed distribution varies depending on the cloud provider. Read more here

How much do you have to change in your training process?

Minimal. Just a few lines of code needed for tracking. Read more

Extensive code and infrastructure changes are required. Check out a few examples here

Does it integrate with the training process via CLI/YAML/Client library?

Yes, through the neptune-client library

Yes, through the kubeflow python client

Does it come with a web UI or is it console-based?
Serverless UI

No

Flexibility, speed, and accessibility chevron
Customizable metadata structure

Yes

Yes

How can you access model metadata?
– gRPC API

No

No

– CLI / custom API

Yes

– REST API

No

Yes

– Python SDK

Yes

Yes

– R SDK

Yes

No

– Java SDK

No

No

– Julia SDK

No

No

Supported operations
– Search

Yes

No

– Update

Yes

No

– Delete

Yes

Limited to deleting entire runs or notebooks

– Download

Yes

No

Distributed training support

Yes

Yes

Pipelining support

Yes

Yes

Logging modes
– Debug

Yes

No

– Asynchronous

Yes

N/A

– Synchronous

Yes

N/A

Live monitoring

Yes

Mobile support

No

No

Webhooks and notifications

No

No

Experiment Tracking

Log and display of metadata chevron
Dataset
– location (path/s3)

Yes

Yes

– hash (md5)

Yes

No

– Preview table

Yes

No

– Preview image

Yes

No

– Preview text

Yes

No

– Preview rich media

Yes

No

– Multifile support

Yes

No

– Dataset slicing support

No

No

Code versions
– Git

Yes

N/A

– Source

Yes

N/A

– Notebooks

Yes

No

Parameters

Yes

Yes

Metrics and losses
– Single values

Yes

Yes

– Series values

Yes

N/A

– Series aggregates (min/max/avg/var/last)

Yes

N/A

Tags

Yes

N/A

Descriptions/comments

Yes

N/A

Rich format
– Images (support for labels and descriptions)

Yes

N/A

– Plots

Yes

Yes

– Interactive visualizations (widgets and plugins)

Yes

N/A

– Video

Yes

No

– Audio

Yes

No

– Neural Network Histograms

No

N/A

– Prediction visualization (tabular)

No

Yes

– Prediction visualization (image)

No

No

– Prediction visualization (image – interactive confusion matrix for image classification)

No

NA

– Prediction visualization (image – overlayed prediction masks for image segmentation)

No

NA

– Prediction visualization (image – overlayed prediction bounding boxes for object detection)

No

NA

Hardware consumption
– CPU

Yes

No

– GPU

Yes

No

– TPU

No

No

– Memory

Yes

No

System information
– Console logs (Stderr, Stdout)

Yes

No

– Error stack trace

Yes

No

– Execution command

No

No

– System details (host, user, hardware specs)

Yes

No

Environment config
– pip requirements.txt

Yes

No

– conda env.yml

Yes

No

– Docker Dockerfile

Yes

No

Files
– Model binaries

Yes

Yes

– CSV

Yes

N/A

– External file reference (s3 buckets)

Yes

Yes

Comparing experiments chevron
Table format diff

Yes

Yes

Overlayed learning curves

Yes

Parameters and metrics
– Groupby on experiment values (parameters)

Yes

No

– Parallel coordinates plots

Yes

No

– Parameter Importance plot

No

No

– Slice plot

No

No

– EDF plot

No

No

Rich format (side by side)
– Video

No

No

– Audio

No

No

– Plots

No

No

– Interactive visualization (HTML)

No

Yes

– Text

Yes

No

– Neural Network Histograms

No

No

– Prediction visualization (tabular)

Yes

No

– Prediction visualization (image, video, audio)

No

No

Code
– Git

No

No

– Source files

No

No

– Notebooks

Yes

No

Environment
– pip requirements.txt

No

No

– conda env.yml

No

No

– Docker Dockerfile

No

No

Hardware
– CPU

Yes

No

– GPU

Yes

No

– Memory

Yes

No

System information
– Console logs (Stderr, Stdout)

Yes

No

– Error stack trace

Yes

No

– Execution command

No

No

– System details (host, owner)

Yes

No

Data versions
– Location

Yes

No

– Hash

Yes

No

– Dataset diff

Yes

No

– External reference version diff (s3)

No

No

Files
– Models

No

No

– CSV

No

No

Custom compare dashboards
– Combining multiple metadata types (image, learning curve, hardware)

Yes

No

– Logging custom comparisons from notebooks/code

No

No

– Compare/diff of multiple (3+) experiments/runs

Yes

No

Organizing and searching experiments and metadata chevron
Experiment table customization
– Adding/removing columns

Yes

No

– Renaming columns in the UI

Yes

No

– Adding colors to columns

Yes

No

– Displaying aggregate (min/max/avg/var/last) for series like training metrics in a table

Yes

N/A

– Automagical column suggestion

Yes

No

Experiment filtering and searching
– Searching on multiple criteria

Yes

N/A

– Query language vs fixed selectors

Query language

N/A

– Saving filters and search history

Yes

No

Custom dashboards for a single experiment
– Can combine different metadata types in one view

Yes

N/A

– Saving experiment table views

Yes

No

– Logging project-level metadata

Yes

N/A

– Custom widgets and plugins

No

Yes

Tagging and searching on tags

Yes

N/A

Nested metadata structure support in the UI

Yes

Reproducibility and traceability chevron
One-command experiment re-run

No

No

Experiment lineage
– List of datasets used downstream

No

– List of other artifacts (models) used downstream

No

– Downstream artifact dependency graph

No

Reproducibility protocol

Limited

N/A

Is environment versioned and reproducible

Yes

N/A

Saving/fetching/caching datasets for experiments

No

N/A

Collaboration and knowledge sharing chevron
Sharing UI links with project members

Yes

N/A

Sharing UI links with external people

Yes

N/A

Commenting

N/A

Interactive project-level reports

No

N/A

Model Registry

Model versioning chevron
Code versions (used for training)

Yes

N/A

Environment versions

No

N/A

Parameters

Yes

Yes

Dataset versions

Yes

N/A

Results (metrics, visualizations)

Yes

N/A

Explanations (SHAP, DALEX)

N/A

Model files (packaged models, model weights, pointers to artifact storage)

Yes

N/A

Model lineage and evaluation history chevron
Models/experiments created downstream

No

History of evaluation/testing runs

No

N/A

Support for continuous testing

No

N/A

Users who created a model or downstream experiments

No

N/A

Access control, model review, and promoting models chevron
Main stage transition tags (develop, stage, production)

Yes

N/A

Custom stage tags

No

N/A

Locking model version and downstream runs, experiments, and artifacts

No

N/A

Adding annotations/comments and approvals from the UI

N/A

Model compare (current vs challenger etc)

Limited

N/A

Compatibility audit (input/output schema)

No

N/A

Compliance audit (datasets used, creation process approvals, results/explanations approvals)

No

N/A

CI/CD/CT compatibility chevron
Webhooks

No

N/A

Model accessibility

No

N/A

Support for continuous testing

No

N/A

Integrations with CI/CD tools

No

N/A

Model searching chevron
Registered models

Yes

N/A

Active models

Yes

N/A

By metadata/artifacts used to create it

Yes

N/A

By date

Yes

N/A

By user/owner

Yes

N/A

By production stage

Yes

N/A

Search query language

Yes

N/A

Model packaging chevron
Native packaging system

No

N/A

Compatibility with packaging protocols (ONNX, etc)

No

Yes

One model one file or flexible structure

No

N/A

Integrations with packaging frameworks

No

Yes

Integrations and Support

Languages chevron
Java

No

No

Julia

No

No

Python

Yes

Yes

R

No

REST API

No

Yes

Model training chevron
Catalyst

Yes

No

CatBoost

No

No

fastai

Yes

No

FBProphet

Yes

No

Gluon

No

No

HuggingFace

Yes

No

H2O

No

No

LightGBM

Yes

No

Paddle

No

No

PyTorch

Yes

No

PyTorch Ignite

No

PyTorch Lightning

Yes

Yes

Scikit Learn

Yes

No

Skorch

Yes

No

Spacy

No

No

Spark MLlib

No

No

Statsmodel

No

No

TesorFlow / Keras

Yes

Yes

XGBoost

Yes

Yes

Hyperparameter Optimization chevron
Hyperopt

No

Yes

Keras Tuner

No

Optuna

Yes

Yes

Ray Tune

No

No

Scikit-Optimize

Yes

Model visualization and debugging chevron
DALEX

Yes

No

Netron

No

No

SHAP

No

No

TensorBoard

Yes

IDEs and Notebooks chevron
JupyterLab and Jupyter Notebook

Yes

Yes

Google Colab

Yes

Yes

Deepnote

Yes

No

AWS SageMaker

Yes

No

Data versioning chevron
DVC

Yes

No

Orchestration and pipelining chevron
Airflow

No

No

Argo

No

Yes

Kedro

Yes

No

Kubeflow

No

N/A

ZenML

Yes

Yes

Experiment tracking tools chevron
MLflow

No

Sacred

Yes

No

TensorBoard

Yes

CI/CD chevron
GitHub Actions

No

Gitlab CI

No

No

CircleCI

No

No

Travis

No

No

Jenkins

No

No

Model serving chevron
Seldon

No

Yes

Cortex

No

No

Databricks

No

No

Model versioning chevron
Seldon

No

No

Fiddler.ai

No

No

Arthur.ai

No

No

LLMs chevron
LangChain

No

No

This table was updated on 4 November 2022. Some information may be outdated.
Report outdated information here.

What are the key advantages of Neptune then?

Get started with Neptune
  • Neptune is a paid hosted metadata store with the main focus on experiment tracking and model registry
  • Kubeflow is an open-source project created to enable easier deployment of ML workflows on Kubernetes

Neptune and Kubeflow are not mutually exclusive. In fact, Neptune can serve as a great solution for experiment management and model registry inside the Kubeflow Pipelines.

Check how to start using it.

See these features in action

1

Sign up to Neptune and install client library

pip install neptune
2

Track experiments

import neptune

run = neptune.init_run()
run["params"] = {
    "lr": 0.1, "dropout": 0.4
}
run["test_accuracy"] = 0.84
3

Register models

import neptune

model = neptune.init_model()
model["model"] = {
    "size_limit": 50.0,
    "size_units": "MB",
}
model["model/signature"].upload(
    "model_signature.json")
decor

Thousands of ML people already chose their tool

quote
Previously used tensorboard and azureml but Neptune is hugely better. In particular, getting started is really easy; documentation is excellent, and the layout of charts and parameters is much clearer.
Simon Mackenzie AI Engineer and Data Scientist
quote
(…) thanks for the great tool, has been really useful for keeping track of the experiments for my Master’s thesis. Way better than the other tools I’ve tried (comet / wandb).

I guess the main reason I prefer neptune is the interface, it is the cleanest and most intuitive in my opinion, the table in the center view just makes a great deal of sense. I like that it’s possible to set up and save the different view configurations as well. Also, the comparison is not as clunky as for instance with wandb. Another plus is the integration with ignite, as that’s what I’m using as the high-level framework for model training.
Klaus-Michael Lux Data Science and AI student, Kranenburg, Germany
quote
This thing is so much better than Tensorboard, love you guys for creating it!
Dániel Lévai Junior Researcher at Rényi Alfréd Institute of Mathematics in Budapest, Hungary
quote
While logging experiments is great, what sets Neptune apart for us at the lab is the ease of sharing those logs. The ability to just send a Neptune link in slack and letting my coworkers see the results for themselves is awesome. Previously, we used Tensorboard + locally saved CSVs and would have to send screenshots and CSV files back and forth which would easily get lost. So I’d say Neptune’s ability to facilitate collaboration is the biggest plus.
Greg Rolwes Computer Science Undergraduate at Saint Louis University
quote
Such a fast setup! Love it:)
Kobi Felton PhD student in Music Information Processing at Télécom Paris
quote
For me the most important thing about Neptune is its flexibility. Even if I’m training with Keras or Tensorflow on my local laptop, and my colleagues are using fast.ai on a virtual machine, we can share our results in a common environment.
Víctor Peinado Senior NLP/ML Engineer

It only takes 5 minutes to integrate Neptune with your code

Don’t overthink it

    Contact with us

    This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

    * - required fields