Blog » ML Tools » How to Make your Sacred Projects Easy to Share and Collaborate on

How to Make your Sacred Projects Easy to Share and Collaborate on

Gone are the days of drawn-out spreadsheets detailing chaotic ML experiments. With platforms such as Sacred, you can log experiments with peace of mind. Sacred does the good work for you: it keeps track of your parameters, model architectures, dataset changes, training jobs, artifacts, metrics, meta, debugging, and more. 

In essence, Sacred is a Python module that lets you configure, organize, log, and reproduce experiments. This way, you can: 

  • easily manage your experiment parameters
  • play around with settings for your experiments
  • save configurations for individual runs in a MongoDB database
  • replicate results

Sacred comes with a robust command-line interface, where you can change parameters and carry out different experiments and variants. Through its “observers,” it can log a variety of details about experiments – like dependencies, configurations, machines used, or results. This info can go to a database or another experiment tracking tool. 

Is there a catch? Well, for all of its worth, Sacred doesn’t have a thorough UI or key experiment tracking features. 

This is where Omniboard comes in. Omniboard is Sacred’s twin – you can’t use one without the other. Omniboard is a NodeJS server that connects to Sacred’s database and visualizes experiments + metrics/logs. You can: 

  • Access experiment management features through a table or list view
  • Compare experiments 
  • Review detailed data regarding experiments, such as metric graphs, source files, artifacts, or git hash/version control info. 

The combination of Sacred and Omniboard can be powerful, but they lack valuable functionalities that can be crucial for large ML teams. For example:

  • Features for detailed team collaboration 
  • UI that shows notebook versioning or notebook auto snapshots
  • UI that can save experiment views or group experiments
  • UI that can scale to millions of runs
  • Dedicated user support

However, don’t lose hope yet. You can easily get all these features, along with the Sacred functionalities you know and love, with a Neptune integration. 


READ NEXT
👉 Sacred + Omniboard vs Neptune – Comparison
👉 The Best Sacred + Omniboard Alternatives


Introducing Neptune

Logging metadata

Neptune is a lightweight ML experiment management tool. It’s flexible and easy to integrate with different workflows. Your teammates can use any ML libraries and platforms, share results, and collaborate on a single Neptune dashboard. If you don’t want to deploy it on your own hardware, you can use their web platform. 

Neptune’s main features are: 

  • Experiment Management: keep track of all your team’s experiments, also tag, filter, group, sort, and compare them 
  • Notebook versioning and diffing: compare two notebooks or checkpoints in the same notebook; similarly to source code, you can do a side-by-side comparison 
  • Team Collaboration: add comments, mention teammates, and compare experiment results

For more details, see -> Neptune vs. Sacred + Omniboard

Neptune + Sacred Integration 

Omniboard is the popular front-end for Sacred – however, as we discussed above, it lacks many key features, especially for collaboration. Neptune, on the other hand, lets you stay with the Sacred logging API, while providing its sleek, intuitive UI. It’s the more functional alternative to Omniboard. 

When integrating with Sacred, Neptune replaces the MongoDB backend with its own Observer. This way, you don’t have to set up your database. All your data can be logged to the cloud or locally, however you like. 

Now lets walk through how to sync them: 

Neptune Setup (skip if you already have a Neptune account)

  1. Sign up for a Neptune AI account first. It’s free for individuals and non-organizations, and you get a generous 100 GB of storage. 
  2. Get your API token by clicking the top right menu. 
Neptune Sacred get started
  1. Create a NEPTUNE_API_TOKEN environment variable and run it in your console. export NEPTUNE_API_TOKEN=’your_api_token’
  2. Create a project. In your Projects dashboard, click “New Project” and fill in the following information. Pay attention to the privacy settings!
Neptune Sacred new project

Integration

First, you will need to install your neptune-client

pip install neptune-client

Make sure to create an experiment as well: 

ex = Experiment('iris_rbf_svm')

Then, pass the experiment object as the first argument: 

from neptunecontrib.monitoring.sacred import NeptuneObserver
ex.observers.append(NeptuneObserver(api_token='ANONYMOUS',
                                   project_name='shared/sacred-integration'))

Make sure to replace ‘ANONYMOUS’ with your own API token (see above!) and project name. 

After that, simply run your experiments through Sacred as you normally would. Now, they will train in Neptune!

Neptune Sacred monitoring

Collaborate with Neptune

Your experiment metadata should now be stored in Neptune, and you can view it in your experiment dashboard:

Sacred Neptune integration

You can customize the dashboard by adding tags and grouping experiments with custom filters. 

Neptune lets you share ML experiments simply by sending a link. It can be:

Mlflow experiment links
Mlflow experiment charts
MLflow group of experiments
MLflow diagnosis plots

Neptune also comes with workspaces, a central hub where you can manage projects, users, and subscriptions; there are individual and team workspaces. 

In the team workspace, team members can browse the content that’s related to their assigned role. You can assign various roles in projects and workspaces. In a team workspace, you can invite people either as admin or member, each with different privileges. 

Workplace settings can be changed in the workspace name on the top bar: 

Neptune workspaces

Under the Overview, Projects, People and Subscription tabs, you can see workplace settings: 

Neptune workplace settings

There are three roles in a project: owner, contributor, and viewer. Depending on the role, users can run experiments, create notebooks, modify previous stored data, etc. 

For more details, see -> User Management

Learning more about Neptune

As you can see, Neptune complements Sacred nicely. You can easily replace Omniboard and add the Neptune Observer to unlock more functionality. Neptune can act as a central hub to manage your experiments and collaborate on them with your team. 

If you want to learn more about Neptune, check out the official documentation. If you want to try it out, create your account and start tracking your machine learning experiments with Neptune.


NEXT STEPS

How to get started with Neptune in 5 minutes

1. Create a free account
Sign up
2. Install Neptune client library
pip install neptune-client
3. Add logging to your script
import neptune.new as neptune

run = neptune.init('Me/MyProject')
run['params'] = {'lr':0.1, 'dropout':0.4}
run['test_accuracy'] = 0.84
Try live notebook

The Best Sacred + Omniboard Alternatives

Read more
Best tools featured

15 Best Tools for Tracking Machine Learning Experiments

Read more
Experiment tracking in project management

How to Fit Experiment Tracking Tools Into Your Project Management Setup

Read more
MLOps

MLOps: What It Is, Why it Matters, and How To Implement It (from a Data Scientist Perspective)

Read more