Delphi : A Distributed, Multi-Model, Self-Learning Platform for Machine Learning on Big Data

Technology #17197

Questions about this technology? Ask a Technology Manager

Download Printable PDF

Una-May O'Reilly
Computer Science and Artificial Intelligence Laboratory, MIT
External Link (
Kalyan Veeramachaneni
Computer Science and Artificial Intelligence Laboratory, MIT
External Link (
Will Drevo
Computer Science and Artificial Intelligence Laboratory, MIT
External Link (
Managed By
Daniel Dardani
MIT Technology Licensing Officer
Patent Protection

Distributed, Multi-Model, Self-Learning Platform For Machine Learning

US Patent Pending US 2016-0132787

Model Recommendation System For Machine Learning

Provisional Patent Application Filed


A wide range of possible applications exist for this technology, and the inventors have validated the system using examples from fields as disparate as education, healthcare and web forums. Any company using predictive analysis and machine learning can use this system.

Problem Addressed

While big data and data mining systems that can analyze data exist, a key bottleneck in the field of machine learning/data mining and enterprise model search is the inability of systems to predict or suggest algorithms for a new data set based on experiences with prior data sets. This provides a dataset-size invariant approach to accelerating model search that provides significant improvement over state of the art grid search and hyperparameter search methods.


Delphi is a distributed system that runs multiple machine learning algorithms on many machines across a cloud to find predictive models of optimal efficiency. It is designed to learn from its experience across a multitude of data sets and act as a recommender system that points out the most promising approach. It uses Bayesian optimization and Multi-Armed Bandit techniques for inter-model selection. It manages big data by breaking it down into smaller chunks to find optimal models on these, and then combining them into a higher performance meta-model while making use of the previously created models to enable knowledge transfer in modeling. The software has been tested with multiple data sets.


  • ¬†Algorithms predicted by Delphi shown to be 3-5% more accurate, and up to 20% more in some cases
  • ¬†Shortens the algorithm design process, which would usually take a human team up to a year.