import neptune
run = neptune.init_run()
run["params"] = {
"batch_size": 64,
"dropout": 0.4,
}
run["model"].upload(
"my_model.pkl")
Track, compare, and share your models in one place
ānot all over the place
- Skip days of debugging disorganized models ā and reach production-ready stage faster
- Avoid long and messy model handovers ā and get instant access to model artifacts
Lightweight component that works with any MLOps stack
Track & compare
Log model metadata from anywhere in your pipeline. See and compare results in the web app.
Cut down the time you spend debugging. Monitor training live. Develop production-ready models quicker.
- Any framework
- Any metadata type
- Advanced comparison
- Live training monitoring
- Reproducibility
import neptune
# Connect to Neptune and create a run
run = neptune.init_run()
# Log hyperparameters
run["parameters"] = {
"batch_size": 64,
"dropout":0.5,
"optimizer": {"type":"SGD", "learning_rate": 0.001},
}
# Log dataset versions
run["data/train_version"].track_files("train/images")
# Log the training process
for iter in range(100):
run["train/accuracy"].append(accuracy)
# Log test metrics and charts
run["test/f1_score"] = test_score
run["test/confusion_matrix"].upload(fig)
# Log model weights and versions
run["model/weights"].upload("my_model.pkl")
# Stop logging to your run
run.stop()from neptune.integrations.pytorch import NeptuneLogger
run = neptune.init_run()
neptune_callback = NeptuneLogger(run=run, model=model)from neptune.integrations.tensorflow_keras import NeptuneCallback
run = neptune.init_run()
neptune_cbk = NeptuneCallback(run=run)
model.fit(callbacks=[neptune_callback], ...)from lightning.pytorch.loggers import NeptuneLogger
neptune_logger = NeptuneLogger()
trainer = Trainer(max_epochs=10, logger=neptune_logger)
trainer.fit(my_model, my_dataloader)import neptune.integrations.sklearn as npt_utils
run = neptune.init_run()
parameters = {
"n_estimators": 120,
"learning_rate": 0.12,
"min_samples_split": 3,
"min_samples_leaf": 2,
}
gbc = GradientBoostingClassifier(**parameters)
gbc.fit(X_train, y_train)
run["cls_summary"] = npt_utils.create_classifier_summary(
gbc, X_train, X_test, y_train, y_test
)from neptune.integrations.lightgbm import NeptuneCallback, create_booster_summary
run = neptune.init_run()
neptune_callback = NeptuneCallback(run=run)
params = {
"boosting_type": "gbdt",
"objective": "multiclass",
"num_class": 10,
"metric": ["multi_logloss", "multi_error"],
"num_leaves": 21,
"learning_rate": 0.05,
"max_depth": 12,
}
# Train the model
gbm = lgb.train(
params,
lgb_train,
num_boost_round=200,
valid_sets=[lgb_train, lgb_eval],
valid_names=["training", "validation"],
callbacks=[neptune_callback],
)
run["lgbm_summary"] = create_booster_summary(
booster=gbm,
log_trees=True,
list_trees=[0, 1, 2, 3, 4],
log_confusion_matrix=True,
y_pred=y_pred,
y_true=y_test,
)from neptune.integrations.xgboost import NeptuneCallback
run = neptune.init_run()
neptune_callback = NeptuneCallback(run=run, log_tree=[0, 1, 2, 3])
params = {
"eta": 0.7,
"gamma": 0.001,
"max_depth": 9,
"objective": "reg:squarederror",
"eval_metric": ["mae", "rmse"],
}
xgb.train(
params=params,
dtrain=dtrain,
num_boost_round=num_round,
evals=evals,
callbacks=[neptune_callback],
)import neptune.integrations.optuna as npt_utils
run = neptune.init_run()
neptune_callback = npt_utils.NeptuneCallback(run)
study = optuna.create_study(direction="maximize")
study.optimize(objective, n_trials=20, callbacks=[neptune_callback])from neptune_airflow import NeptuneLogger
with DAG(...) as dag:
def task(**context):
neptune_logger = NeptuneLogger()
...
# Get the Neptune run from the current task
# and log metadata manually
with neptune_logger.get_run_from_context(
context=context, log_context=log_context
) as run:
...
run["checkpoint"].upload_files("my_model.h5")
run["score"] = 0.97
for epoch in range(100):
run["train/accuracy"].append(acc)
run["model/parameters"] = {
"lr": 0.2,
"optimizer": {"name": "Adam", "momentum": 0.9},
}run["train/images"].track_files("./datasets/images")run["matplotlib-fig"].upload(fig)
for name in misclassified_images_names:
run["misclassified_images"].append(File("misclassified_image.png"))run["visuals/altair-fig"].upload(File.as_html(fig))run["video"].upload("/path/to/video-file.mp4")run = neptune.init_run(capture_hardware_metrics=True)run = neptune.init_run(
source_files=["**/*.py", "config.yaml"],
dependencies="infer",
)
Share & collaborate
Collaborate with the team and across the organization
Our built-in collaborative features make light work of your teamwork. Centralized data means all your team members can access your experiments and models and share the results in one place.
- Create dashboards and save views
- Share persistent UI links
- Manage users and projects
- Add your entire org
Version & store
Version production-ready models for a zero-headache model handover
Time to ship your model to prod? Enable instant access to model artifacts for ML Engineers, so they can test, package, and deploy models to production even faster.
- Version models
- Version external artifacts
- Access and share models
model_version = neptune.init_model_version(with_id="FACE-DET-42")
model_version["model/signature"].download()
Code examples, videos, projects gallery, and other resources
Yes, you can deploy Neptune on-premises and other answers
-
Read more about our deployment options here.
But in short, yes, you can deploy Neptune on your on-prem infrastructure or in your private cloud.Ā
It is a set of microservices distributed as a Helm chart that you deploy on Kubernetes.Ā
If you donāt have your own Kubernetes cluster deployed, our installer will set up a single-node cluster for you.Ā
As per infrastructure requirements, you need a machine with at least 8 CPUs,Ā 32GB RAM, and 1TB SSD storage.
Read the on-prem documentation if you’re interested, or talk to us (support@neptune.ai) if you have questions.
If you have any trouble, our deployment engineers will help you all the way.
-
Yes, you can just reference datasets that sit on your infrastructure or in the cloud.Ā
For example, you can have your datasets on S3 and just reference the bucket.Ā
run[“train_dataset”].track_files(“s3://datasets/train.csv”)
Neptune will save the following metadata about this dataset:Ā
- version (hash),Ā
- location (path),Ā
- size,Ā
- folder structure, and contents (files)
Neptune never uploads the dataset, just logs the metadata about it.Ā
You can later compare datasets or group experiments by dataset version in the UI.
-
Short version. People choose Neptune when:
- They donāt want to maintain infrastructure (including autoscaling, backups etc.),
- They keep scaling their projects (and get into thousands of runs),
- They collaborate with a team (and want user access, multi-tenant UI etc.).
For the long version, read this full feature-by-feature comparison.
-
Short version. People choose Neptune when:
- They want to pay a reasonable price for the experiment tracking solution,
- They want a super flexible tool (customizable logging structure, dashboards, works great with time series ML),
- They want a component for experiment tracking and model versioning, NOT an end-to-end platform (WandB has HPO, orchestration, model deployment, etc. We integrate with best-in-class tools in the space).
For the long version, read this full feature-by-feature comparison.
-
It depends on what āmodel monitoringā you mean.Ā
As we talk to teams, it seems that “model monitoring” means six different things to three different people:Ā
- (1) Monitor model performance in production: See if the model performance decays over time, and you should re-train it
- (2) Monitor model input/output distribution: See how the distribution of input data, features, and predictions distribution change over time?
- (3) Monitor model training and re-training: See learning curves, trained model predictions distribution, or confusion matrix during training and re-training
- (4) Monitor model evaluation and testing: log metrics, charts, prediction, and other metadata for your automated evaluation or testing pipelines
- (5) Monitor hardware metrics: See how much CPU/GPU or Memory your models use during training and inference
- (6) Monitor CI/CD pipelines for ML: See the evaluations from your CI/CD pipeline jobs and compare them visually
So when looking at tooling landscape and Neptune:
- Neptune does (3) and (4) really well, but we see teams use it for (5) and (6). We don’t do (1) and (2), but we integrate with tools for that.
- Prometheus + Grafana is really good at (5), but people use it for (1) and (2).
- WhyLabs or Arize are really good at (1) and (2).