The Union of Intersections (UoI) Method for Interpretable Data-Driven Discovery and Prediction: Data Science Lecture Series

Lecture | February 17 | 1:10-2:30 p.m. | 190 Doe Library

 Berkeley Institute for Data Science

The increasing size and complexity of scientific data could dramatically enhance basic scientific discovery and prediction for applications. Realizing this potential requires novel statistical analysis algorithms that are both interpretable and predictive. We introduce the Union of Intersections (UoI) method, a flexible, modular, and scalable paradigm applicable to a variety of machine learning problems. UoI satisfies the bicriteria of accurate recovery of a small number of interpretable features while maintaining high-quality prediction accuracy. This talk will introduce the core concepts of UoI, summarize theoretical results on its properties, and demonstrate its superiority for linear regression, classification, random forests, and non-negative matrix factorizations. These results suggest UoI could improve interpretation and prediction in data-driven discovery across scientific fields.