Time: 4:30-5:30 p.m.
Date: Wednesday, January 28, 2026
Speaker: Keyon Vafa, Postdoctoral Fellow, Harvard University
Title: Assessing AI’s Implicit World Models
Abstract: Modern generative models like large language models (LLMs) can perform very well on benchmarks yet fail on seemingly related real-world tasks. How can we determine whether generative models actually do what we think they do? This talk will treat this challenge as a statistical inference problem. I will present methods designed to make inferences about the structural understanding — or implicit “world models” — of generative models. I will propose different ways to formalize the concept of a world model, develop practical tests based on these formalizations, and apply them across empirical domains. Developing reliable inferences about model capabilities more broadly would offer new ways to assess, and ultimately improve, the efficacy of generative models. Throughout the talk, I will show that while these problems involve modern computational methods, they require combining computational techniques with classical statistical ideas.
Bio: Keyon Vafa is a Harvard Data Science Initiative postdoctoral fellow at Harvard University. His research focuses on bridging prediction and structural recovery in generative models, with applications spanning both general machine learning settings and the social sciences. Keyon completed his PhD in computer science from Columbia University, where he was an NSF GRFP Fellow and the recipient of the Morton B. Friedman Memorial Prize for excellence in engineering. He has organized the NeurIPS 2024 Workshop on Behavioral Machine Learning and the ICML 2025 Workshop on Assessing World Models, and serves on the Early Career Board of the Harvard Data Science Review.