Ecole d'ingénieur et centre de recherche en Sciences du numérique

Enabling scalable stochastic gradient-based inference for Gaussian processes by employing the Unbiased LInear System SolvEr (ULISSE)

Filippone, Maurizio; Engler, Raphael

ICML 2015, International Conference on Machine Learning, July 6-11, 2015, Lille, France

In applications of Gaussian processes where quantification of uncertainty is of primary interest, it is necessary to accurately characterize the posterior distribution over covariance parameters. This paper proposes an adaptation of the Stochastic Gradient Langevin Dynamics algorithm to draw samples from the posterior distribution over covariance parameters with negligible bias and without the need to compute the marginal likelihood. In Gaussian process regression, this has the enormous advantage that stochastic gradients can be computed by solving linear systems only. A novel unbiased linear systems solver based on parallelizable covariance matrix-vector products is developed to accelerate the unbiased estimation of gradients. The results demonstrate the possibility to enable scalable and exact (in a Monte Carlo sense) quantifi- cation of uncertainty in Gaussian processes without imposing any special structure on the covariance or reducing the number of input vectors.

Document Arxiv Bibtex

Titre:Enabling scalable stochastic gradient-based inference for Gaussian processes by employing the Unbiased LInear System SolvEr (ULISSE)
Type:Conférence
Langue:English
Ville:Lille
Pays:FRANCE
Date:
Département:Data Science
Eurecom ref:4793
Copyright: © 2015 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
Bibtex: @inproceedings{EURECOM+4793, year = {2015}, title = {{E}nabling scalable stochastic gradient-based inference for {G}aussian processes by employing the {U}nbiased {LI}near {S}ystem {S}olv{E}r ({ULISSE})}, author = {{F}ilippone, {M}aurizio and {E}ngler, {R}aphael}, booktitle = {{ICML} 2015, {I}nternational {C}onference on {M}achine {L}earning, {J}uly 6-11, 2015, {L}ille, {F}rance}, address = {{L}ille, {FRANCE}}, month = {07}, url = {http://www.eurecom.fr/publication/4793} }
Voir aussi: