Graduate School and Research Center in Digital Sciences

Bayesian adaptive filtering

Sadiki, Tayeb


While adaptive filtering is in principle intended for tracking non-stationary systems, most adaptive filtering algorithms have been designed for converging to a fixed unknown filter. When actually confronted with a non-stationary environment, they possess just only one parameter (stepsize, forgetting factor) to adjust their tracking capability. Virtually the only existing optimal approach is the Kalman filter, in which the time-varying optimal filter is modeled as a vector AR(1) process. The Kalman filter is in practice never applied as an adaptive filter because of its complexity and large number of unknown parameters in its state-space (AR(1)) model. This motivated our work in this thesis to look for practical techniques to take advantage of the Kalman optimality with reduced complexity to make this approach applicable. Also to propose different methods for the estimation of the large number of parameters. In the first part we begin by an introductory state of the art review which includes the existent algorithms on adaptive filtering (LMS and LRS). These algorithms experience performances limitation in terms of tracking and convergence in non-stationary environments. This motivates our work to propose more efficient techniques for such Bayesian techniques. Our proposed methods take into consideration a priori information about the system variations such as the PDP, Doppler bandwidth, ...etc. We thus propose two different approaches, the first one is based on Wiener Filtering (WF) while the other one is based on Kalman Filtering (KF). The proposed technique is based on modeling the optimal adaptive filter coefficients as a stationary vector process, in particular as a AR(1) model. Optimal adaptive filtering with such a state model becomes Kalman filtering. The complexity of the resulting algorithm is and in order to reduce this complexity we propose a diagonal AR(1) based model, of complexity which is comparable to RLS complexity. For the AR(1) model parameters estimation, we propose an adaptive version of the EM algorithm with complexity limited to for the EM part. The proposed parameters estimation method leads to linear prediction on reconstructed optimal filter correlations, and hence a meaningful approximation/estimation compromise. To further reduce the initial adaptive EM-Kalman algorithm complexity, we develop a second approach based on component-wise EM-Kalman (This technique is of complexity which is comparable to LMS complexity). In the second part of the thesis we consider the problem of window optimization issues in recursive Least-Squares adaptive filtering and tracking. We consider tracking of an optimal filter modeled as a stationary vector process. We interpret the Recursive Least-Squares (RLS) adaptive filtering algorithm as a filtering operation on the optimal filter process and the intantaneous gradient noise (induced by the measurement noise). The filtering operation carried out by the RLS algorithm depends on the window used in the least-squares criterion. To arrive at a recursive LS algorithm requires that the window impulse response can be expressed recursively (output of an IIR filter). In practice, only two popular window choices exist (with each one tuning parameter): the exponential weighting (W-RLS) and the rectangular window (SWC-RLS). However, the rectangular window can be generalized at a small cost for the resulting RLS algorithm to a window with three parameters (GSW-RLS) instead of just one, encompassing both SWC- andW-RLS as special cases. Since the complexity of SWC-RLS essentially doubles with respect toW-RLS, it is generally believed that this increase in complexity allows for some improvement in tracking performance. We show that, with equal estimation noise, W-RLS generally outperforms SWC-RLS in causal tracking, with GSW-RLS still performing better, whereas for non-causal tracking SWC-RLS is by far the best (with GSW-RLS not being able to improve). When the window parameters are optimized for causal tracking MSE, GSW-RLS outperforms W-RLS which outperforms SWC-RLS. We also derive the optimal window shapes for causal and non-causal tracking of arbitrary variation spectra. It turns outs that W-RLS is optimal for causal tracking of AR(1) parameter variations whereas SWC-RLS if optimal for non-causal tracking of integrated white jumping parameters, all optimal filter parameters having proportional variation spectra in both cases.

Document Bibtex

Title:Bayesian adaptive filtering
Department:Communication systems
Eurecom ref:2170
Copyright: © Université de Nice. Personal use of this material is permitted. The definitive version of this paper was published in Thesis and is available at :
Bibtex: @phdthesis{EURECOM+2170, year = {2007}, title = {{B}ayesian adaptive filtering}, author = {{S}adiki, {T}ayeb}, school = {{T}hesis}, month = {03}, url = {} }
See also: