Functional priors for bayesian neural networks through wasserstein distance minimization to Gaussian processes

The Bayesian treatment of neural networks dictates that a prior distribution is considered

over the weight and bias parameters of the network. The non-linear nature of the model

implies that any distribution of the parameters has an unpredictable effect on the distribution of the function output. Gaussian processes offer a rigorous framework to define prior distributions over the space of functions. Our proposal is to impose such functional priors on well-established architectures of neural networks by means of minimising the Wasserstein distance between samples of stochastic processes. Early experimental results demonstrate the potential of functional priors for Bayesian neural networks.