How Cradle Achieved Experiment Tracking and Data Security Goals With Self-Hosted Neptune
Cradle is a biotechnology company leveraging ML and generative AI to accelerate protein engineering. Their software platform enables customers in pharmaceuticals, materials science, sustainable food production, etc., to design novel proteins. A protein can be anything from an antibody that needs to bind better while making sure it doesn’t solicit an undesired immune response to an enzyme that needs to catalyze a chemical reaction while still maintaining high expression values.
By combining computational methods with experimental data, Cradle optimizes protein structures and functions, dramatically reducing the time and cost of traditional protein engineering processes.
The challenge
Cradle uses experiment tracking both for internal projects, focused on research and development of novel methods for protein optimization, and for production projects, where models are trained ad-hoc for proteins of interest of their customers.
They first opted for Weights & Biases, which served this purpose quite well. However, the self-hosted deployment turned out to be complicated and rather time-consuming.
So they decided to look for alternatives, and after analyzing the potential options, they decided to go with Neptune.
Unlocked self-hosted deployment
Reducing attack vectors to the absolute minimum is crucial for some of Cradle’s larger customers, particularly in the pharma industry. The intellectual property value of some of the therapeutic proteins being optimized can be extremely high. So Cradle treats the security and privacy of its customers’ data as one of its top priorities. The company even formed a partnership with Google Cloud to provide an unprecedented level of security in this area.
For this reason, Cradle doesn’t store any customer data outside of its premises, hence the experiment tracker also had to be deployed on the company’s infrastructure.
Weights & Biases fell short here; it was mainly focused on AWS, which didn’t mesh well with Cradle’s use of Google Cloud (GCP).
Neptune’s support for on-premises deployment allowed the company to comply with stringent data security requirements and maintain key client relationships.
Thankfully, Neptune’s on-premise installation offered the flexibility and adjustability we required. The process was well-prepared, and their engineers were incredibly helpful, answering all our questions and even guiding us through a simpler deployment approach.
Hassle-free migration from WandB to Neptune
Since Cradle already had several active projects, it was important that switching to Neptune would be as quick and painless as possible. They didn’t want customer projects to be delayed because of the switch.
Migrating from Weights & Biases to Neptune was a smooth and straightforward process for Cradle. Both tools are quite similar in functionality, making the transition easy to manage.
For Cradle’s model training and inference, which utilized PyTorch Lightning, the change was as simple as updating the tracker from WandB to Neptune.
As an extra, the team added some custom helpers to the Neptune Logger for PyTorch Lighting, which was also quick and not complicated.
Cradle’s team decided not to migrate historical data but focused on preserving important artifacts. Since Weights and Biases’ artifacts were backed by GCS paths, the team created a database to reference these paths without recreating the artifacts in Neptune. This approach streamlined the process and avoided unnecessary complications.
Interested in migrating historical data from WandB to Neptune?
Overall, the entire migration process, including on-prem installation and migrating the code base to Neptune, was done in 4 weeks and was primarily performed by one engineer. This ensured their workflow continued uninterrupted, allowing the rest of the team to focus on their core tasks without downtime.
Unexpected gains from switching to Neptune
While the self-hosted deployment was the main driver for searching for an alternative experiment tracking solution, several aspects of Neptune stood out when the team started using it.
The way of handling artifacts
Tracking artifacts is one of the key functionalities of an experiment tracker for the Cradle team. They use it for tracing data and model provenance, and providing context for model evaluation.
The previous tracker used its own GCS bucket for artifact storage, renaming files, and creating reference links rather than directly accessing the customer’s GCS. This approach required users to consult the UI or make API calls to locate the actual file path. It all added unnecessary complexity.
Neptune’s approach is more straightforward and allows for direct linking to artifacts in the customer’s storage without the need for intermediate steps.
Using direct file paths instead of a reference system simplified the team’s workflow. It allows them to use artifacts directly with their Flyte system for provenance and serialization without needing to import Neptune’s client library and translate artifact names into paths every time.
More flexible Python client library
One of the highlights for the team was Neptune’s Python client library. With their previous experiment tracker, they found the API somewhat limiting, especially for more complex tasks. It often felt like it was designed with simplicity in mind, which was great for quick onboarding but problematic for more advanced use cases. The reliance on a single object that tracks everything further constrained their work.
In contrast, Neptune offers a lower-level API, which provides greater flexibility and control. You can also log to multiple objects at once. This allowed the team to execute tasks more precisely and maintain cleaner, more readable code. Neptune’s API design felt more robust, allowing the team to accomplish their goals with fewer workarounds and more direct implementations.
More responsive web app
Another advantage was the responsiveness of Neptune’s user interface. While the previous solution had a slightly more polished look at the time, it often struggled with performance issues, particularly when loading pages.
Neptune, on the other hand, delivered a much faster and smoother user experience, even with a few thousands of runs across multiple workspaces. This saved the team valuable time and reduced frustration during daily operations.
The results
- After moving to the self-hosted version of the experiment tracking component of their stack, Cradle was able to meet the stringent security requirements of some of its largest clients.
- Direct artifact handling reduced file tracking complexity and improved integration with existing systems like Flyte.
- Neptune’s flexible Python library and responsive web app increased team efficiency in executing complex tasks and accessing experiment data.
Product resource 
For us, such deployment was too difficult and time-consuming in the previous solution. We could achieve that with Neptune, and it allowed us to close important deals that had stringent security requirements.
