Particle Physics Seminar: Universal Phenomena in Singular Bayesian Learning

Abstract:

The asymptotic behaviour of Bayesian learning in singular models is governed by two invariants of the singularity: the real log-canonical threshold λ, which controls the rate of posterior concentration, and the singular fluctuation ν, which controls the gap between training and generalization loss. The RLCT is known for many models; ν is known for almost none.

This thesis studies several aspects of these asymptotics – effective regularity, an input–output symmetry of the invariants, the relevance of the prior, and explicit asymptotic formulas for ν – primarily in the reduced-rank regression model Y = BAX + ε, a singular model that is well understood and simple enough to study in detail. We also ask whether the same phenomena persist in non-linear models, and identify the mechanisms that determine when they do. Results are proved where possible; the remaining structural claims are conjectures supported by Gibbs sampling experiments. 

Seminar Organizer: Dr. Igor Korover