Member-only storyDaniel WarfieldFollowTowards Data Science--ShareIn this post we will explore why experiment orchestration is important, existing orchestration solutions, how to build your own orchestrator with MongoDB, and why that might be beneficial in some use cases.Who is this useful for? Anyone trying to fit models to data; and consequently needs a way to organize those experiments.How advanced is this post? The idea of orchestration is fairly simple, and is accessible to virtually any skill level. The example should be accessible to backend developers or data scientists trying to branch out.Pre-requisites: A working understanding of core networking principles, like databases and servers, as well as core data science concepts, like hyperparameters.Code: Full code can be found here. Note: this repo is a WIP at the time of writing this article.By “experiment orchestration” I am referring to a wide number of tasks consisting of the same general concept. The most common form of experiment orchestration is hyperparameter sweeps: where, given some range of hyperparameter values, you want to sweep through that range and find the best hyperparameter set for a given modeling problem. The organization of these types of planned experiments is generally referred to as orchestration.Simple sweeps get the job done most of the time, but as modeling problems become more complex it is common to have more complex experiments. You may find yourself needing to experiment with multiple model types, each with their own hyperparameter space, across multiple datasets.For instance, I’m currently conducting research on the performance of different modeling strategies within non-homogenous modeling applications. I’m not interested in “what is the best set of hyperparameters to solve a particular problem”, but rather “how do multiple model types, each with their own hyperparameter…----Towards Data ScienceData Scientist, Educator, Artist, and Writer.Daniel WarfieldinTowards Data Science--2Damian GilinTowards Data Science--22Giuseppe ScalamognainTowards Data Science--17Daniel WarfieldinTowards Data Science--1Damian GilinTowards Data Science--22Haifeng Li--Yugank Aman--David Shapiro--6Muhammad Imran--Michal SzudejkoinThe Writing Cooperative--3HelpStatusWritersBlogCareersPrivacyTermsAboutText to speechTeams