BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Laboratoire de Mathématiques de Versailles - ECPv6.15.20//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:Laboratoire de Mathématiques de Versailles
X-ORIGINAL-URL:https://lmv.math.cnrs.fr
X-WR-CALDESC:Évènements pour Laboratoire de Mathématiques de Versailles
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-Robots-Tag:noindex
X-PUBLISHED-TTL:PT1H
BEGIN:VTIMEZONE
TZID:Europe/Paris
BEGIN:DAYLIGHT
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
TZNAME:CEST
DTSTART:20240331T010000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
TZNAME:CET
DTSTART:20241027T010000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
TZNAME:CEST
DTSTART:20250330T010000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
TZNAME:CET
DTSTART:20251026T010000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
TZNAME:CEST
DTSTART:20260329T010000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
TZNAME:CET
DTSTART:20261025T010000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=Europe/Paris:20250214T100000
DTEND;TZID=Europe/Paris:20250214T110000
DTSTAMP:20260420T133044
CREATED:20240904T164342Z
LAST-MODIFIED:20250214T125212Z
UID:13094-1739527200-1739530800@lmv.math.cnrs.fr
SUMMARY:PS : Benjamin Dupuis (INRIA et ENS Paris) : Data-dependent Uniform Generalization Bounds
DESCRIPTION:We study data-dependent uniform generalization bounds by approaching the problem from a PAC-Bayesian perspective. We first apply the PAC-Bayesian framework on « random sets » in a rigorous way\, where the training algorithm is assumed to output a data-dependent hypothesis set after observing the training data. This approach allows us to prove data-dependent bounds\, which can be applicable in numerous contexts. To highlight the power of our approach\, we consider several applications of these new techniques.\nFirst\, we prove uniform bounds over the trajectories of continuous Langevin dynamics and stochastic gradient Langevin dynamics. These results provide novel information about the generalization properties of noisy algorithms.\nSecond\, we focus on the geometric terms appearing in our generalization bounds and use it to (1) simplify and recover existing fractal-based generalization bounds and (2) develop new topological bounds that are better adapted to discrete-time stochastic optimizers compared to the previous fractal approach. \nLinks to the papers:\n– https://arxiv.org/abs/2404.17442\n– https://arxiv.org/abs/2407.08723\n– https://arxiv.org/abs/2302.02766
URL:https://lmv.math.cnrs.fr/evenenement/ps-benjamin-dupuis-inria-et-ens-paris/
LOCATION:Bâtiment Fermat\, salle 4205
CATEGORIES:Séminaire PS
END:VEVENT
BEGIN:VEVENT
DTSTART;TZID=Europe/Paris:20250214T113000
DTEND;TZID=Europe/Paris:20250214T123000
DTSTAMP:20260420T133044
CREATED:20240904T164618Z
LAST-MODIFIED:20250207T112138Z
UID:13098-1739532600-1739536200@lmv.math.cnrs.fr
SUMMARY:PS : Umut Simsekli (INRIA et ENS Paris) : Implicit Compressibility of Overparametrized Neural Networks Trained with Heavy-Tailed SGD
DESCRIPTION:Neural network compression has been an increasingly important subject\, not only due to its practical relevance\, but also due to its theoretical implications\, as there is an explicit connection between compressibility and generalization error. In this talk\, I will present a simple modification for SGD\, such that the outputs of the algorithm will be provably compressible without making any nontrivial assumptions. We will consider a one-hidden-layer neural network trained with SGD\, and show that if we inject additive heavy-tailed noise to the iterates at each iteration\, for any compression rate\, there exists a level of overparametrization such that the output of the algorithm will be compressible with high probability. To achieve this result\, we make two main technical contributions: (i) we prove a “propagation of chaos” result for a class of heavy-tailed stochastic differential equations\, and (ii) we derive error estimates for their Euler discretization. Our experiments suggest that the proposed approach not only achieves increased compressibility with various models and datasets\, but also leads to robust test performance under pruning\, even in more realistic architectures that lie beyond our theoretical setting. The talk is based on the following article: https://arxiv.org/pdf/2306.08125.pdf
URL:https://lmv.math.cnrs.fr/evenenement/ps-umut-simsekli-inria-et-ens-paris/
LOCATION:Bâtiment Fermat\, salle 4205
CATEGORIES:Séminaire PS
END:VEVENT
END:VCALENDAR