SAVED - Space alternating variational estimation for sparse Bayesian learning with parametric dictionaries

In this paper, we address the fundamental problem of sparse signal recovery in a Bayesian framework, where the received signal is a multi-dimensional tensor. We further consider the problem of dictionary learning, where the tensor observations are assumed to be generated from a Khatri-Rao structured dictionary matrix multiplied by the sparse coefficients. We consider a Bayesian approach using variational Bayesian (VB) inference. VB allows one to obtain analytical approximations to the posterior distributions of interest even when an exact inference of these distributions is intractable. We propose a novel fast algorithm called space alternating variational estimation with dictionary learning (SAVED), which is a version of VB(-SBL) pushed to the scalar level. Similarly, as for SAGE (space-alternating generalized expectation maximization) compared to EM, the componentwise approach of SAVED compared to sparse Bayesian learning (SBL) renders it less likely to get stuck in bad local optima and its inherent damping (more cautious progression) also leads to typically faster convergence of the non-convex optimization process. Simulation results show that the proposed algorithm has a faster convergence rate and lower mean squared error (MSE) compared to the alternating least squares based method for tensor decomposition.