PS : Ismaël Castillo (Sorbonne Université) : Deep horseshoe Gaussian processes

Chargement Évènements

« Tous les Évènements

  • Cet évènement est passé

PS : Ismaël Castillo (Sorbonne Université) : Deep horseshoe Gaussian processes

17 janvier / 11:30 - 12:30

Algorithms modeling a possibly `deep’ structure in data have gained considerable popularity in recent years. Indeed, data sitting on a high-dimensional space can often be described by a hidden structure of much smaller « effective dimension ». A popular class of methods in this context is that of deep neural networks, e.g. with ReLU activation function. Another possibility is to use so-called deep Gaussian processes as prior distributions within a Bayesian approach.

In this talk I will first review a few results on Gaussian processes (GPs) used as priors in nonparametric function estimation problems, showing that GPs (combined with an  extra rescaling random variable) yield optimal convergence rates for posterior distributions, rates that are moreover adaptive to the `smoothness’ of the unknown function.

To achieve rates that are also adaptive to `structure’,  one may consider deep Gaussian Processes, namely compositions of Gaussian processes. Recently, Finocchio and Schmidt-Hieber (preprint, 2021) showed that there exists a well-chosen prior distribution consisting of a deep Gaussian process coupled with priors modeling both the smoothness and a `variable selection’ step, that achieves a near-optimal posterior rate, adaptive to both regularity and structure.

In this work, we introduce deep horseshoe Gaussian processes: each Gaussian process in the composition features a `lengthscale’ parameter along each dimension, parameter which itself gets a horseshoe prior distribution. We show that this horseshoe prior enables simultaneous adaptation to both smoothness and structure at optimal rates, without the need of a variable selection prior. The overall prior is then fairly close to the versions of deep GP priors that have been deployed in practice.

This is joint work with Thibault Randrianarisoa (Bocconi University, Milano).

PS : Ismaël Castillo (Sorbonne Université) : Deep horseshoe Gaussian processes


17 janvier
Heure :
11:30 - 12:30
Catégorie d’Évènement:


Bâtiment Fermat, salle 4205


Alexis Devulder
Julien Worms