Time: 4:30-5:30 p.m.
Date: Monday, January 26, 2026
Speaker: Licong Lin
Title: Towards a Statistical Theory of Contrastive Learning in Modern AI
Abstract: Contrastive learning has emerged as a central paradigm for representation learning in modern AI, with examples ranging from single-modal vision encoders like SimCLR to multimodal systems like CLIP that combine vision and language. Despite its widespread empirical success, a theoretical understanding of why contrastive learning produces broadly transferable representations remains limited. In this talk, I will introduce the concept of approximate sufficient statistics, a generalization of the classical sufficient statistics, and show that near-minimizers of the contrastive loss yield representations that are approximately sufficient, making them adaptable to diverse downstream tasks. I will first describe results for single-modal, augmentation-based contrastive learning, showing that contrastively learned encoders can be adapted to downstream tasks, with performance depending on their sufficiency and the augmentation-induced error. I will then extend the framework to multimodal settings and discuss implications for downstream tasks such as zero-shot classification, conditional diffusion models, and vision-language models. Together, these results provide a unified statistical perspective on why contrastively learned representations can be effectively adapted across tasks and modalities.
Bio: Licong Lin is a fifth-year Ph.D. candidate in the Department of Statistics at UC Berkeley, advised by Song Mei and Peter Bartlett. His research focuses on the theoretical foundations and algorithms for AI. He uses and extends tools from statistical learning theory, high-dimensional statistics, and optimization to study the statistical foundations of architectures, algorithms, and phenomena in modern AI (e.g., Transformers, in-context learning, contrastive learning, scaling laws), and to develop mathematically motivated algorithms for AI alignment (e.g., LLM unlearning).