Compare
Neptune vs DagsHub
vs
Neptune
DagsHub
Commercial Requirements
Commercial requirements
Standalone component or a part of a broader ML platform?
Standalone component. ML metadata store that focuses on experiment tracking and model registry
Standalone tool
Is the product available on-premises and / or in your private/public cloud?
Only public cloud support for the free plan. On-prem/private cloud installation only for paid plans. Read more about their plans here
Is the product delivered as commercial software, open-source software, or a managed cloud service?
Managed cloud service
Managed cloud service
What is the pricing model?
SLOs / SLAs: Does the vendor provide guarantees around service levels?
Support: Does the vendor provide 24Ć7 support?
SSO, ACL: does the vendor provide user access management?
General Capabilities
Setup
What are the infrastructure requirements?
No special requirements other than having the neptune-client installed and access to the internet if using managed hosting. Check here for infrastructure requirements for on-prem deployment
How much do you have to change in your training process?
Minimal. Just a few lines of code needed for tracking. Read more
Minimal. Just a few lines of code need to be added for basic tracking
Does it integrate with the training process via CLI/YAML/Client library?
Yes, through the neptune-client library
Yes, through the Git and DVC CLI
Serverless UI
No
No
Flexibility, speed, and accessibility
How can you access model metadata?
ā gRPC API
No
No
ā REST API
No
No
Supported operations
Logging modes
Experiment Tracking
Log and display of metadata
Dataset
ā Dataset slicing support
No
No
Code versions
Metrics and losses
Rich format
ā Neural Network Histograms
No
No
ā Prediction visualization (tabular)
No
No
ā Prediction visualization (image)
No
No
ā Prediction visualization (image ā interactive confusion matrix for image classification)
No
NA
ā Prediction visualization (image ā overlayed prediction masks for image segmentation)
No
NA
ā Prediction visualization (image ā overlayed prediction bounding boxes for object detection)
No
NA
Hardware consumption
ā TPU
No
No
System information
ā Execution command
No
No
Environment config
Files
Comparing experiments
Parameters and metrics
ā Parameter Importance plot
No
No
ā Slice plot
No
No
ā EDF plot
No
No
Rich format (side by side)
ā Video
No
No
ā Audio
No
No
ā Plots
No
No
ā Interactive visualization (HTML)
No
No
ā Neural Network Histograms
No
No
ā Prediction visualization (image, video, audio)
No
No
Code
Environment
ā pip requirements.txt
No
No
ā conda env.yml
No
No
ā Docker Dockerfile
No
No
Hardware
System information
ā Execution command
No
No
Data versions
Files
ā Models
No
No
Custom compare dashboards
ā Logging custom comparisons from notebooks/code
No
No
Organizing and searching experiments and metadata
Experiment table customization
ā Renaming columns in the UI
Yes
No
ā Adding colors to columns
Yes
No
Experiment filtering and searching
ā Query language vs fixed selectors
Query language
Fixed selectors
Custom dashboards for a single experiment
ā Custom widgets and plugins
No
No
Reproducibility and traceability
One-command experiment re-run
No
No
Experiment lineage
ā List of other artifacts (models) used downstream
No
No
ā Downstream artifact dependency graph
No
No
Collaboration and knowledge sharing
Interactive project-level reports
No
No
Model Registry
Model versioning
Model lineage and evaluation history
Models/experiments created downstream
No
No
History of evaluation/testing runs
No
No
Support for continuous testing
No
No
Users who created a model or downstream experiments
No
No
Access control, model review, and promoting models
Custom stage tags
No
No
Locking model version and downstream runs, experiments, and artifacts
No
No
Model compare (current vs challenger etc)
Limited
No
Compatibility audit (input/output schema)
No
No
Compliance audit (datasets used, creation process approvals, results/explanations approvals)
No
No
CI/CD/CT compatibility
Model accessibility
No
No
Support for continuous testing
No
No
Integrations with CI/CD tools
No
No
Model searching
Model packaging
Native packaging system
No
No
Compatibility with packaging protocols (ONNX, etc)
No
No
One model one file or flexible structure
No
No
Integrations with packaging frameworks
No
No
Integrations and Support
Model training
Hyperparameter Optimization
Model visualization and debugging
IDEs and Notebooks
Model serving
Seldon
No
No
Cortex
No
No
Databricks
No
No
Model versioning
Seldon
No
No
Fiddler.ai
No
No
Arthur.ai
No
No
This table was updated on 2 April 2022. Some information may be outdated.
Report outdated information here.
Report outdated information here.
What are the key advantages of Neptune then?
- Customizable metadata structure
- Possibility to log many metadaty types, inlcuding rich format
- Possibility to combine multiple metadata types into dashboards
- Support for distributed training
- Hardware consumption monitoring
See these features in action
1
Sign up to Neptune and install client library
pip install neptune2
Track experiments
import neptune
run = neptune.init_run()
run["params"] = {
"lr": 0.1, "dropout": 0.4
}
run["test_accuracy"] = 0.843
Register models
import neptune
model = neptune.init_model()
model["model"] = {
"size_limit": 50.0,
"size_units": "MB",
}
model["model/signature"].upload(
"model_signature.json")
Thousands of ML people already chose their tool
Previously used tensorboard and azureml but Neptune is hugely better. In particular, getting started is really easy; documentation is excellent, and the layout of charts and parameters is much clearer.
Simon Mackenzie
AI Engineer and Data Scientist
(ā¦) thanks for the great tool, has been really useful for keeping track of the experiments for my Masterās thesis. Way better than the other tools Iāve tried (comet / wandb).
I guess the main reason I prefer neptune is the interface, it is the cleanest and most intuitive in my opinion, the table in the center view just makes a great deal of sense. I like that itās possible to set up and save the different view configurations as well. Also, the comparison is not as clunky as for instance with wandb. Another plus is the integration with ignite, as thatās what Iām using as the high-level framework for model training.
Klaus-Michael Lux
Data Science and AI student, Kranenburg, Germany
I guess the main reason I prefer neptune is the interface, it is the cleanest and most intuitive in my opinion, the table in the center view just makes a great deal of sense. I like that itās possible to set up and save the different view configurations as well. Also, the comparison is not as clunky as for instance with wandb. Another plus is the integration with ignite, as thatās what Iām using as the high-level framework for model training.
This thing is so much better than Tensorboard, love you guys for creating it!
DƔniel LƩvai
Junior Researcher at RƩnyi AlfrƩd Institute of Mathematics in Budapest, Hungary
While logging experiments is great, what sets Neptune apart for us at the lab is the ease of sharing those logs. The ability to just send a Neptune link in slack and letting my coworkers see the results for themselves is awesome. Previously, we used Tensorboard + locally saved CSVs and would have to send screenshots and CSV files back and forth which would easily get lost. So Iād say Neptuneās ability to facilitate collaboration is the biggest plus.
Greg Rolwes
Computer Science Undergraduate at Saint Louis University
Such a fast setup! Love it:)
Kobi Felton
PhD student in Music Information Processing at TƩlƩcom Paris
For me the most important thing about Neptune is its flexibility. Even if Iām training with Keras or Tensorflow on my local laptop, and my colleagues are using fast.ai on a virtual machine, we can share our results in a common environment.
VĆctor Peinado
Senior NLP/ML Engineer
It only takes 5 minutes to integrate Neptune with your code
Donāt overthink it