Subhabrata Sen is an Assistant Professor of Statistics at Harvard University. Prior to Harvard, he was a Schramm postdoctoral fellow at Microsoft Research New England and MIT. He obtained his PhD from Stanford Statistics in 2017. His research lies at the intersection of applied probability, statistics and machine learning. His research interests include high-dimensional and nonparametric statistics, random graphs and inference on networks. Subhabrata has received the Probability Dissertation Award for his thesis, an AMS Simons Travel grant, and an honorable mention at the Bernoulli Society New Researcher Award (2018). He has been a long term visitor at the Simons Institute for the Theory of Computing in Fall 2021 and Fall 2022.
Talk: Mean-field approximations for high-dimensional Bayesian Regression
Abstract: Variational approximations provide an attractive computational alternative to MCMC-based strategies for approximating the posterior distribution in Bayesian inference. The Naive Mean-Field (NMF) approximation is the simplest version of this strategy—this approach approximates the posterior in KL divergence by a product distribution. There has been considerable progress recently in understanding the accuracy of the NMF approximation in high-dimensional statistical models. However, it is expected that in some high-dimensional settings, the NMF approximation should be grossly inaccurate. Advanced meanfield techniques (e.g. Bethe approximation, expectation propagation), which improve the NMF approximation by incorporating additional corrections, are expected to be superior in these regimes. Unfortunately, a rigorous understanding of these advanced mean-field methods is severely limited at present. In this talk, we will focus on a specific high-dimensional problem—Bayes linear regression with an iid gaussian design, when the number of features and datapoints are both large and comparable. The NMF approximation is conjectured to be inaccurate for this model. Instead, the Thouless-Anderson-Palmer (TAP) approximation from statistical physics (an advanced mean-field method) is conjectured to provide a tight approximation. We will establish the validity of the TAP approximation under a uniform spherical prior on the regression coefficients.
This is based on joint work Jiaze Qiu (Harvard University).