Slow fading channel selection: A restless multi-armed bandit formulation

We deal with a multi-access wireless network in which transmitters dynamically select a frequency band to communicate on. The slow fading channel attenuations follow an autoregressive model. In the single user case, we formulate this selection problem as a restless multi-armed bandit problem and we propose two strategies to dynamically select a band at each time slot. Our objective is to maximize the SNR in the long run. Each of these strategies is close to the optimal strategy in different regimes. In the general case with several users, we formulate the problem as a stochastic game with uncountable state space, where the objective is the SINR. Then we propose two strategies to approximate the best response policy for one user when the other users' strategy is fixed.