Graduate School and Research Center in Digital Sciences

Accelerating deep Gaussian processes inference with arc-cosine kernels

Cutajar, Kurt; Bonilla, Edwin V; Michiardi, Pietro; Filippone, Maurizio

NIPS 2016, 30th Annual Conference on Neural Information Processing Systems, Workshop on Bayesian Deep Learning, 10 December 2016, Barcelona, Spain

Deep Gaussian Processes (DGPs) are probabilistic deep models obtained by stacking multiple layers implemented through Gaussian Processes (GPs). Although attractive from a theoretical point of view, learning DGPs poses some significant computational challenges that arguably hinder their application to a wider variety of problems for which Deep Neural Networks (DNNs) are the preferred choice. We aim to bridge the gap between DGPs and DNNs by showing how random feature approximations to DGPs can leverage the key strengths of DNNs, while retaining a probabilistic formulation for accurate quantification of uncertainty. In particular, we show how DGPs with arc-cosine kernels can be approximated by DNNs with Rectified Linear Unit (ReLU) activation functions, leading to competitive performance and faster inference compared to state-of-the-art DGPs inference approaches.

Document Bibtex

Title:Accelerating deep Gaussian processes inference with arc-cosine kernels
Department:Data Science
Eurecom ref:5093
Bibtex: @inproceedings{EURECOM+5093, year = {2016}, title = {{A}ccelerating deep {G}aussian processes inference with arc-cosine kernels}, author = {{C}utajar, {K}urt and {B}onilla, {E}dwin {V} and {M}ichiardi, {P}ietro and {F}ilippone, {M}aurizio}, booktitle = {{NIPS} 2016, 30th {A}nnual {C}onference on {N}eural {I}nformation {P}rocessing {S}ystems, {W}orkshop on {B}ayesian {D}eep {L}earning, 10 {D}ecember 2016, {B}arcelona, {S}pain}, address = {{B}arcelona, {SPAIN}}, month = {12}, url = {} }
See also: