Scalable slice orchestration with DQN agents in beyond 5G networks

Beyond 5G systems are envisioned to host widely diverse services, with high Quality of Service requirements, over a common physical network infrastructure. Network slicing enables this vision by providing customized virtual networks on top of the physical one. These can be flexibly managed to fulfill stringent slice-specific service level agreements (SLAs) and ensure the efficient use of network resources, under timevarying resource demands. In this work, we examine the use of Reinforcement Learning (RL) for dynamic slice orchestration over a multi-domain setup, hosting multiple slices with end-toend SLAs. Tabular RL algorithms can theoretically solve this

problem optimally, even when the resource demand dynamics of slices are unknown. However, due to the combinatorial nature of the problem, the state and action complexity of these algorithms is prohibitive for realistic setups. To this end, we employ a Deep

Q Network (DQN) to approximate the action-value function and therefore reduce complexity. However, since DQN is not suitable for problems with large action spaces, we propose a multi-agent DQN scheme to decompose the action into smaller components.

We finally validate the proposed algorithm using realistic data, and show its scalability as well as the better performance of the derived policies compared to static baseline heuristics.