neptune.ai | The MLOps stack component for experiment tracking

Lightweight component that works with any MLOps stack

Flexible Python library

25+ integrations with ML tools & frameworks

Use SaaS
or deploy on your infra

Track & compare

Log model metadata from anywhere in your pipeline. See and compare results in the web app.

Cut down the time you spend debugging. Monitor training live. Develop production-ready models quicker.

Any framework
Any metadata type
Advanced comparison
Live training monitoring
Reproducibility

Any framework

Any code
PyTorch
Keras
Lightning
scikit-learn
LightGBM
XGBoost
Optuna
Kedro

See all integrations & supported tools

import neptune

# Connect to Neptune and create a run
run = neptune.init_run()

# Log hyperparameters
run["parameters"] = {
    "batch_size": 64,
    "dropout":0.5,
    "optimizer": {"type":"SGD", "learning_rate": 0.001},
}
# Log dataset versions
run["data/train_version"].track_files("train/images")

# Log the training process
for iter in range(100):
    run["train/accuracy"].append(accuracy)

# Log test metrics and charts
run["test/f1_score"] = test_score
run["test/confusion_matrix"].upload(fig)

# Log model weights and versions
run["model/weights"].upload("my_model.pkl")

# Stop logging to your run
run.stop()

from neptune.integrations.pytorch import NeptuneLogger

run = neptune.init_run()
neptune_callback = NeptuneLogger(run=run, model=model)

from neptune.integrations.tensorflow_keras import NeptuneCallback

run = neptune.init_run()
neptune_cbk = NeptuneCallback(run=run)

model.fit(
    x_train,
    y_train,
    epochs=5,
    batch_size=64,
    callbacks=[neptune_cbk],
)

from pytorch_lightning.loggers import NeptuneLogger

neptune_logger = NeptuneLogger()

trainer = Trainer(max_epochs=10, logger=neptune_logger)

trainer.fit(my_model, my_dataloader)

import neptune.integrations.sklearn as npt_utils

run = neptune.init_run()

parameters = {
    "n_estimators": 120,
    "learning_rate": 0.12,
    "min_samples_split": 3,
    "min_samples_leaf": 2,
}

gbc = GradientBoostingClassifier(**parameters)
gbc.fit(X_train, y_train)

run["cls_summary"] = npt_utils.create_classifier_summary(
    gbc, X_train, X_test, y_train, y_test
)

from neptune.integrations.lightgbm import NeptuneCallback, create_booster_summary

run = neptune.init_run()
neptune_callback = NeptuneCallback(run=run)

params = {
    "boosting_type": "gbdt",
    "objective": "multiclass",
    "num_class": 10,
    "metric": ["multi_logloss", "multi_error"],
    "num_leaves": 21,
    "learning_rate": 0.05,
    "max_depth": 12,
}

# Train the model
gbm = lgb.train(
    params,
    lgb_train,
    num_boost_round=200,
    valid_sets=[lgb_train, lgb_eval],
    valid_names=["training", "validation"],
    callbacks=[neptune_callback],
)

run["lgbm_summary"] = create_booster_summary(
    booster=gbm,
    log_trees=True,
    list_trees=[0, 1, 2, 3, 4],
    log_confusion_matrix=True,
    y_pred=y_pred,
    y_true=y_test,
)

from neptune.integrations.xgboost import NeptuneCallback

run = neptune.init_run()
neptune_callback = NeptuneCallback(run=run, log_tree=[0, 1, 2, 3])

params = {
    "eta": 0.7,
    "gamma": 0.001,
    "max_depth": 9,
    "objective": "reg:squarederror",
    "eval_metric": ["mae", "rmse"],
}

xgb.train(
    params=params,
    dtrain=dtrain,
    num_boost_round=num_round,
    evals=evals,
    callbacks=[neptune_callback],
)

import neptune.integrations.optuna as optuna_utils

run = neptune.init_run()
neptune_callback = optuna_utils.NeptuneCallback(run)

study = optuna.create_study(direction="maximize")
study.optimize(objective, n_trials=20, callbacks=[neptune_callback])

kedro neptune init

def report_accuracy(predictions: np.ndarray, test_y: pd.DataFrame,
                    neptune_run: neptune.run.Handler) -> None:
    # ...
    neptune_run["nodes/report/accuracy"] = accuracy

    fig, ax = plt.subplots()
    plot_confusion_matrix(target, predictions, ax=ax)
    neptune_run["nodes/report/confusion_matrix"].upload(fig)

def create_pipeline(**kwargs):
    return Pipeline(
        [# ...
            node(
                report_accuracy,
                ["example_predictions", "example_test_y","neptune_run"],
                None,
                name="report",
            ),
        ]
    )

Any metadata type

Metrics
Parameters
Dataset and model versions
Images
Interactive plots
Videos
Hardware (GPU, CPU, Memory)
Code state

See all supported metadata types

run["score"] = 0.97

for epoch in range(100):
    run["train/accuracy"].append(acc)

run["model/parameters"] = {
    "lr":0.2,
    "optimizer": {"name": "Adam", "momentum": 0.9},
}

run["train/images"].track_files("./datasets/images")

run["matplotlib-fig"].upload(fig)

for name in misclassified_images_names:
    run["misclassified_images"].append(File("misclassified_image.png"))

run["visuals/altair-fig"].upload(File.as_html(fig))

run["video"].upload("/path/to/video-file.mp4")

run = neptune.init_run(capture_hardware_metrics=True)

run = neptune.init_run(source_files=["**/*.py", "config.yaml"])

Advanced comparison

Neptune’s configurable UI allows you to organize and visualize your model metadata however you want.

– Compare metrics and parameters in charts or using parallel coordinates.
– Check differences between runs in a side-by-side table.
– Compare images and datasets.

Cut down the time you spend debugging with a consistent way to compare the effect of different configs on your experiments.

See docs

Live training monitoring

By monitoring your training results in real time, you can react quickly when runs fail or models stop converging.

See docs

Reproducibility

Reproducing your experiments is a no-brainer, too. You can save dataset versions, environment configs, parameters, code, metrics, and model binaries for every experiment you run.

See docs

Share & collaborate

Collaborate with the team and across the organization

Our built-in collaborative features make light work of your teamwork. Centralized data means all your team members can access your experiments and models and share the results in one place.

Create dashboards and save views
Share persistent UI links
Manage users and projects
Add your entire org

Create dashboards and save views

Share persistent UI links

Share every chart, dashboard, table view, or anything else you see in the Neptune app by copying and sending a persistent link. No more sending screenshots on Slack.

See live example

Manage users and projects

Add your entire org

Version & store

Version production-ready models for a zero-headache model handover

Time to ship your model to prod? Enable instant access to model artifacts for ML Engineers, so they can test, package, and deploy models to production even faster.

Version models
Version external artifacts
Access and share models

Version models

Version production-ready models and metadata associated with them in a single place.

See the docs

Version external artifacts

Save hash, location and other model artifact metadata. You don’t have to upload the model to Neptune. Just keep track of the model reference to local or S3-compatible storage.

See the docs

Access and share models

Every model and model version is accessible via the Neptune web app or through the API.

Once you have all the model artifacts, you can deploy your model in your production pipelines or serve it via API.

See the docs

model_version = neptune.init_model_version(with_id="FACE-DET-42")
model_version["model/signature"].download()

0

ML engineers and data scientists

0

Commercial and research teams

0

Projects tracked

Neptune’s UI is highly configurable and the SDK is remarkable, it provided us with the ability to observe and monitor quite complex training ops that no other system would be able to. The dashboards gave us deep insight into our distributed training framework. We came to a resolution to use Neptune because other tools just could not fulfil our requirements.

Yanal Kashou Co-founder & Chief Data Scientist at Elevatus

Gone are the days of writing stuff down on google docs and trying to remember which run was executed with which parameters and for what reasons. Having everything in Neptune allows us to focus on the results and better algorithms.

Andreas Malekos Head of Artificial Intelligence at Continuum Industries

See all case studies

Code examples, videos, projects gallery, and other resources

Frequently asked questions

Yes, you can deploy Neptune on-premises and other answers

Read more about our deployment options here.

But in short, yes, you can deploy Neptune on your on-prem infrastructure or in your private cloud.

It is a set of microservices distributed as a Helm chart that you deploy on Kubernetes.

If you don’t have your own Kubernetes cluster deployed, our installer will set up a single-node cluster for you.

As per infrastructure requirements, you need a machine with at least 8 CPUs, 32GB RAM, and 1TB SSD storage.

Read the on-prem documentation if you’re interested, or talk to us (support@neptune.ai) if you have questions.

If you have any trouble, our deployment engineers will help you all the way.
Yes, you can just reference datasets that sit on your infrastructure or in the cloud.

For example, you can have your datasets on S3 and just reference the bucket.

run[“train_dataset”].track_files(“s3://datasets/train.csv”)

Neptune will save the following metadata about this dataset:
- version (hash),
- location (path),
- size,
- folder structure, and contents (files)
Neptune never uploads the dataset, just logs the metadata about it.

You can later compare datasets or group experiments by dataset version in the UI.
Short version. People choose Neptune when:
- They don’t want to maintain infrastructure (including autoscaling, backups etc.),
- They keep scaling their projects (and get into thousands of runs),
- They collaborate with a team (and want user access, multi-tenant UI etc.).
For the long version, read this full feature-by-feature comparison.
Short version. People choose Neptune when:
- They want to pay a reasonable price and the ability to invite unlimited users for free,
- They want a super flexible tool (customizable logging structure, dashboards, works great with time series ML),
- They want a component for experiment tracking and model registry, NOT an end-to-end platform (WandB has HPO, orchestration, model deployment, etc. We integrate with best-in-class tools in the space).
For the long version, read this full feature-by-feature comparison.
It depends on what “model monitoring” you mean.

As we talk to teams, it seems that “model monitoring” means six different things to three different people:
- (1) Monitor model performance in production: See if the model performance decays over time, and you should re-train it
- (2) Monitor model input/output distribution: See how the distribution of input data, features, and predictions distribution change over time?
- (3) Monitor model training and re-training: See learning curves, trained model predictions distribution, or confusion matrix during training and re-training
- (4) Monitor model evaluation and testing: log metrics, charts, prediction, and other metadata for your automated evaluation or testing pipelines
- (5) Monitor hardware metrics: See how much CPU/GPU or Memory your models use during training and inference
- (6) Monitor CI/CD pipelines for ML: See the evaluations from your CI/CD pipeline jobs and compare them visually
So when looking at tooling landscape and Neptune:
- Neptune does (3) and (4) really well, but we saw teams use it for (5) and (6)
- Prometheus + Grafana is really good at (5), but people use it for (1) and (2)
- WhyLabs or Arize are really good at (1) and (2)

Manage all your model metadata in a single place

Start logging for free See pricing

Deploy Neptune on your infrastructure

Learn more: SaaS or On-prem?

Brainly

Waabi

ReSpo.Vision

Building a Machine Learning Platform

Learnings From Building the ML Platform at Mailchimp

Learn more: SaaS or On-prem?

Brainly

Waabi

ReSpo.Vision

Building a Machine Learning Platform

Learnings From Building the ML Platform at Mailchimp

Track, compare, and share your models in one place
—not all over the place

Lightweight component that works with any MLOps stack

Log model metadata from anywhere in your pipeline. See and compare results in the web app.

Collaborate with the team and across the organization

Version production-ready models for a zero-headache model handover

Code examples, videos, projects gallery, and other resources

Yes, you can deploy Neptune on-premises and other answers

Manage all your model metadata in a single place

Track, compare, and share your models in one place—not all over the place

Lightweight component that works with any MLOps stack

Log model metadata from anywhere in your pipeline. See and compare results in the web app.

Collaborate with the team and across the organization

Version production-ready models for a zero-headache model handover

Code examples, videos, projects gallery, and other resources

Yes, you can deploy Neptune on-premises and other answers

Manage all your model metadata in a single place

Track, compare, and share your models in one place
—not all over the place