Abstract: Integrated Access and Backhaul (IAB) is an emerging technique to enable cost-effective deployment of dense 5G networks that utilize emerging millimeter-wavelength (mmWave) spectrum. Existing heuristic-based network control/management frameworks are not well-suited for the increasing complexity and uncertainty introduced by mmWave IAB. Machine learning (ML) can help automate network control decisions, but its practical deployment faces new system-level challenges in 5G IAB, including accurate simulation-based training, resolving conflicting objectives from heterogeneous network slices, and efficiently collecting observations for run-time decision-making. In this paper, we develop a general framework for effectively deploying reinforcement learning (RL) to control 5G IAB networks. Our framework incorporates a data-driven stochastic simulation scheme to bridge the simulation-to-reality gap, a piecewise reward shaping mechanism to handle competing conflicting performance objectives, and a simple observation selection algorithm to reduce the input size into the RL policy. We validate this framework using real-world network measurements from a mmWave IAB testbed, combined with a large scale ray tracing simulation. Experiments on a set of challenging 5G IAB network control problems demonstrate the effectiveness of our framework to enable practical RL integration into 5G IAB.

Self-supervised reinforcement learning model and MikroTik testbed for dynamic mmWave mesh network..

PDF