By continuing to use this site, you agree to the use of cookies in accordance with our privacy policy.

Science and Mathematics

Mathematics Colloquium: Pratik Patil

January 21, 2025 at 3:30pm5:00pm EST

Carnegie Library, 122

This event has already occurred. The information may no longer be valid.

Pratik Patil from the University of California, Berkeley will be the guest speaker at our upcoming colloquium held on Tuesday, January 21 at 3:30 p.m. in Carnegie 122. We hope to see you there!

Title: Facets of regularization in overparameterized machine learning

Abstract: Modern machine learning often operates in an overparameterized regime in which the number of parameters exceeds the number of observations. In this regime, models can exhibit surprising generalization behaviors: (1) Models can achieve zero training error yet still generalize well (benign overfitting); furthermore, in some cases, adding and tuning explicit regularization can favor no regularization at all (obligatory overfitting). (2) The generalization error can vary non-monotonically with the model or sample size (double/multiple descent). These behaviors challenge classical notions of overfitting and the role of explicit regularization.

In this talk, I will present theoretical and methodological results related to these behaviors, focusing on the concrete case of ridge regularization. First, I will identify conditions under which the optimal ridge penalty is zero (or even negative) and show that standard techniques such as leave-one-out and generalized cross-validation, when analytically continued, remain uniformly consistent for the generalization error and thus yield the optimal penalty, whether positive, negative, or zero. Second, I will introduce a general framework to mitigate double/multiple descent in the sample size based on subsampling and ensembling and show its intriguing connection to ridge regularization. As an implication of this connection, I will show that the generalization error of optimally tuned ridge regression is monotonic in the sample size (under mild data assumptions) and mitigates double/multiple descent. Key to both parts is the role of implicit regularization, either self-induced by the overparameterized data or externally induced by subsampling and ensembling.

The talk will feature joint work with the following collaborators (in surname-alphabetical order): Jin-Hong Du, Arun Kumar Kuchibhotla, Alessandro Rinaldo, Ryan Tibshirani, Yuting Wei. The corresponding papers (in talk-chronological order) are: optimal ridge landscape (https://pratikpatil.io/papers/ridge-ood.pdf), ridge cross-validation (https://pratikpatil.io/papers/functionals-combined.pdf), risk monotonization (https://pratikpatil.io/papers/risk-monotonization.pdf), and ridge equivalences (https://pratikpatil.io/papers/generalized-equivalences.pdf).

This event was published on January 16, 2025.


Event Details