Holden Lee is an assistant professor in Applied Mathematics and Statistics at Johns Hopkins University. He is interested in the interplay between machine learning, probability, and theoretical computer science, with particular focus on building theoretical foundations for generative models and sampling algorithms. He has also worked on learning and control of dynamical systems. Previously, he was a postdoc at Duke and obtained his Ph.D. from Princeton.
Talk: Sampling Approximately Low-Rank Ising Models: MCMC meets Variational Methods
Attend this talk via Zoom // passcode: 917019
Abstract: MCMC and variational inference are two competing paradigms for the problem of sampling from a given probability distribution. In this talk, I'll show how they can work together to give the first polynomial-time sampling algorithm for approximately low-rank Ising models. Sampling was previously known when all eigenvalues of the interaction matrix fit in an interval of length 1; however, a single outlier can cause Glauber dynamics to mix torpidly. Our result covers the case when all but O(1) eigenvalues lie in an interval of length 1. To deal with positive eigenvalues, we use a temperature-based heuristic for MCMC called simulated tempering, while to deal with negative eigenvalues, we define a nonconvex variational problem over Ising models, solved using SGD. Our result has applications to sampling Hopfield networks with a fixed number of patterns, Bayesian clustering models with low-dimensional contexts, and antiferromagnetic/ferromagnetic Ising model on expander graphs. As time allows, I'll discuss more on how this work fits into a broader program of sampling "beyond log-concavity" and further work in this direction.
Based on joint work with Frederic Koehler and Andrej Risteski, https://arxiv.org/abs/2202.08907