Accelerating deep Gaussian processes inference with arc-cosine kernels

Cutajar, Kurt; Bonilla, Edwin V; Michiardi, Pietro; Filippone, Maurizio
NIPS 2016, 30th Annual Conference on Neural Information Processing Systems, Workshop on Bayesian Deep Learning, 10 December 2016, Barcelona, Spain

Deep Gaussian Processes (DGPs) are probabilistic deep models obtained by stacking
multiple layers implemented through Gaussian Processes (GPs). Although
attractive from a theoretical point of view, learning DGPs poses some significant
computational challenges that arguably hinder their application to a wider variety
of problems for which Deep Neural Networks (DNNs) are the preferred choice.
We aim to bridge the gap between DGPs and DNNs by showing how random
feature approximations to DGPs can leverage the key strengths of DNNs, while
retaining a probabilistic formulation for accurate quantification of uncertainty. In
particular, we show how DGPs with arc-cosine kernels can be approximated by
DNNs with Rectified Linear Unit (ReLU) activation functions, leading to competitive
performance and faster inference compared to state-of-the-art DGPs inference
approaches.

Type:
Conférence
City:
Barcelona
Date:
2016-12-10
Department:
Data Science
Eurecom Ref:
5093

PERMALINK : https://www.eurecom.fr/publication/5093