How InstaDeep No Longer Wastes Time Looking for Data With Neptune

InstaDeep is an EMEA leader in decision-making AI products. Collaborating with the biggest names in AI, including Google DeepMind, Intel, and NVIDIA, they’ve built products that solve complex problems across many industries. 

The BioAI team at InstaDeep is where Biology meets Artificial intelligence — pushing the boundaries of medical science through a combination of biology and machine learning expertise. The team reached out to Neptune when building DeepChain™ — InstaDeep’s platform for protein design.

The challenge

The DeepChain™ protein design platform engineers new sequences for protein targets using sophisticated optimization algorithms such as reinforcement learning and evolutionary algorithms. They also leverage Language Models trained on millions of protein sequences and train their in-house language models. Finally, they use machine learning to predict protein structure from sequence.

Building complex software like DeepChain™ involves a lot of research with different moving parts. Customers demand multiple solutions — each requiring new experiments and research. 

Keeping track of all the experiments running for different customers — and remaining productive — was a daunting task for their team.

The BioAI started looking for a solution to make their experiment management more efficient. 

The tool they were searching for had to be: 

Already popular with individual team members — and ticking every box on the list of requirements — BioAI adopted Neptune as their experiment tracking tool of choice.

Here’s how Neptune helped BioAI overcome 3 key challenges.

Visibility for experiment logs

BioAI didn’t have a centralized repository for its enormous amount of data, so finding specific experiment results became a huge challenge. 

Engineers spent more time figuring out where the results were rather than doing the actual research. Researchers would take ages to compare the results of previous runs because they’d have to search through the entire log file. A productive workflow, it wasn’t.

Neptune gave BioAI easy visibility on experiment logs by centralizing all experiment runs, indexing, and organizing them. For the first time, the team could view details of various experiment runs, search for specific runs through their metadata and tags, and see visualizations related to particular experiments.

I think Neptune does one thing very well, which is get your logs and charts right where and when you need them… The search feature of looking for runs and using the tags for the runs is very good as well. The idea of tagging experiment runs is very useful.

Nicolas Lopez Carranza DeepChain and BioAI Lead at InstaDeep

Easy shareability of results

There’s no easy way to share results when logs are all over the place. The essential collaboration between researchers working on many experiments at once turns into an uphill struggle. 

BioAI’s team couldn’t share results from Tensorboard without clunky workarounds.  And when researchers have to share experiments by exposing their URLs — and worry about security along the way — collaboration suddenly becomes much less attractive.   

Neptune provided BioAI’s team with a password-protected and easily shareable link to experiment results without any extra configuration. Collaboration on experiments is now straightforward — with no additional hassle. They can also collaborate on research through Neptune’s feature for comparing runs and experiment metrics.

I like that Neptune does not get in your way – it is not very intrusive. It also does very well with the comparison of runs, sharing, and working collaboratively.

Nicolas Lopez Carranza DeepChain and BioAI Lead at InstaDeep

Eliminating DevOps and infrastructure config

Using Tensorboard to manage their experiments, the BioAI team had to spin up and manage the infrastructure before they could visualize experiment results. 

If you have had to configure any software infrastructure before, you know this is no easy task — even for an operations engineer. For researchers, it’s plain painful — and a massive time-suck for their team.

Neptune eliminated BioAI’s need to configure infrastructure for logging experiment results by providing a managed solution for the team.

No more DevOps needed for logging. No more starting VMs just to look at some old logs. No more moving data around to compare TensorBoards.

Nicolas Lopez Carranza DeepChain and BioAI Lead at InstaDeep

The results

1. Zero time looking for data: BioAI’s team now spends almost no time searching for results and metadata on experiments. All thanks to the visibility they get with Neptune’s centralized experiment management dashboard. “No time spent looking for the data. It’s always there, available, and displayed the way we want.” – says Nicolas.
2. Research productivity through the roof: Neptune eliminated the obstacles BioAI’s team faced managing a lot of experiments. Productivity — and their ability to easily collaborate — has massively improved.

Thanks to Nicolas Lopez Carranza for his help in creating this case study!

Read also

Product resource

Dashboards for Different Stages of the ML Project

Read more Product resource

Reinforcement Learning Example Project

Read more Product resource

How Brainly Enabled Tracking Metadata from SageMaker Pipelines

Read more Related article

Building a Machine Learning Platform [Definitive Guide]

Read more

Back to Resources Back to Customers