Model selection for Bayesian autoencoders

Tran, Ba-Hien; Rossi, Simone; Milios, Dimitrios; Michiardi, Pietro; Bonilla, Edwin V.; Filippone, Maurizio
NeurIPS 2021, 35th Conference on Neural Information Processing Systems, 6-14 December 2021 (Virtual Event)

We develop a novel method for carrying out model selection for Bayesian autoencoders (BAEs) by means of prior hyper-parameter optimization. Inspired by the common practice of type-II maximum likelihood optimization and its equivalence to Kullback-Leibler divergence minimization, we propose to optimize the distributional sliced-Wasserstein distance (DSWD) between the output of the autoencoder and the empirical data distribution. The advantages of this formulation are that we can estimate the DSWD based on samples and handle high-dimensional problems. We carry out posterior estimation of the BAE parameters via stochastic gradient Hamiltonian Monte Carlo and turn our BAE into a generative model by fitting a flexible Dirichlet mixture model in the latent space. Consequently, we obtain a powerful alternative to variational autoencoders, which are the preferred choice in modern applications of autoencoders for representation learning with uncertainty. We evaluate our approach qualitatively and quantitatively using a vast experimental campaign on a number of unsupervised learning tasks and show that, in smalldata regimes where priors matter, our approach provides state-of-the-art results, outperforming multiple competitive baselines.


HAL
Type:
Conference
Date:
2021-12-05
Department:
Data Science
Eurecom Ref:
6588
Copyright:
© NIST. Personal use of this material is permitted. The definitive version of this paper was published in NeurIPS 2021, 35th Conference on Neural Information Processing Systems, 6-14 December 2021 (Virtual Event) and is available at :

PERMALINK : https://www.eurecom.fr/publication/6588