Hierarchical multi-agent RL TE in SD-WAN based cloud edge continuum interconnection

The emergence of IoT and the rise of latencysensitive services are pushing computation from centralised clouds toward a Cloud-Edge Continuum (CEC), where services of the same application are deployed across different CEC nodes, with services requiring low latency closer to the user. This transition introduces new challenges, especially interconnecting these CEC nodes and deployed services dynamically to meet their service level agreements (SLAs). In this paper, we address dynamic Traffic Engineering (TE) in SD-WAN in order to maximise services’ QoS requirements fulfilment. We introduce a solution based on Hierarchical Multi-Agent Reinforcement Learning (H-MARL) for dynamically routing service flows through appropriate overlay links that maximise the QoS and reduce the network cost. Simulation results show the efficiency of the proposed solution in dropping overall latency and improving QoS fulfilment by about 10% compared with a flat single-layer MARL.