Performance bounds for cochannel interference cancellation within the current GSM standard

The performance of maximum-likelihood sequence estimation (MLSE) is bounded (and often approximated well) by the matched filter bound (MFB). In this paper, we first show how the linearized GMSK modulation of GSM can be reformulated to have a real symbol constellation, leading to a two-channel aspect due to the in-phase (I) and in-quadrature (Q) components of the received signal. More channels can be added by oversampling and/or the use of multiple sensors (antennas, polarizations). Given this model, we present the MFB for MLSE single-user detection in the presence of interferers that are modeled as colored Gaussian noise. The MLSE is assumed to employ noise correlation information that in general may differ from the true one (e.g. by simplifying spatio-temporal correlations to spatial correlations only). One of the conclusions is that properly taking into account the spatio-temporal correlation of the interference leads to much improved performance (MFB) compared to purely spatial approaches. Another conclusion is that the excess bandwidth in GMSK provides some useful channel diversity.