Coordinating multiple unmanned aerial vehicle (UAV) relays requires joint control of UAV mobility and transmission power, dynamic association between UAVs and device-to-device (D2D) pairs, and management of inter-UAV proximity risk. This paper presents Safe-CTDE, a heuristic-augmented, value-based centralized training with decentralized execution (CTDE) framework for multi-UAV relay networks. A parameter-shared Q-network is evaluated separately for each UAV using its local observation, while a centralized state-value network provides system-level temporal-difference (TD) feedback during training. UAV–device-pair association is handled separately by a capacity-aware Hungarian scheduler in which each D2D pair is assigned to at most one UAV, while each UAV may serve multiple pairs subject to a training-dependent capacity. The local Q-target combines an agent-specific shaping reward with a clipped global TD signal. During Q-guided action selection, deterministic motion/power priors and a one-step proximity-aware safety filter bias the selected action away from locally predicted separation violations; uniform ε-random exploration itself is not filtered. Reward weights follow a curriculum during the first part of training and a feedback-based adaptation mechanism thereafter. In the reported single-run experiment with five UAVs and five D2D pairs, the final Safe-CTDE training trajectory reaches approximately 69 kbps aggregate throughput with a proximity surrogate of approximately 0.30. The aligned non-learning reference curves span approximately 57–81 kbps in throughput and 0.19–0.49 in the adopted proximity metric. These results are interpreted as a proof-of-concept intermediate operating point among the tested behaviors rather than as statistical superiority or a hard collision-avoidance guarantee.

Multi-UAV relay network, multi-agent reinforcement learning, CTDE, trajectory control, user association, action masking, safety-aware learning