DESIGN OF DEEP REINFORCEMENT LEARNING AGENTS FOR DYNAMIC QUALITY OF SERVICE ADAPTATION IN MAVLINK-MQTT ARCHITECTURES

The integration of Unmanned Aerial Vehicles (UAVs) into the Internet of Drones (IoD) paradigm requires scalable and efficient communication architectures. While the combination of the Micro Air Veichle Link (MAVlink) protocol with the lightweight Message Queuing Telemetry Transport (MQTT) protocol addresses interoperability issues and provides a viable solution for large-scale connectivity, managing bandwidth consumption and latency under dynamic network conditions remains a critical challenge. This thesis addresses these challenges by developing autonomous Deep Reinforcement Learning (DRL) agents capable of dynamically optimizing the MQTT Quality of Service (QoS) level based on real-time network conditions. The primary contributions of this work include the definition of a data-driven framework for dynamic QoS adaptation and an unsupervised network state ab- straction based on Isolation Forest, which enables the detection of silent network failures often missed by static heuristic approaches. Two distinct offline agents are trained using Conservative Q-Learning (CQL) to ensure robust policy learning from historical data: a Reliability-First agent, designed to minimize packet loss in response to physical network degradation, and a Content-Adaptive agent, which implements a novel content-aware strategy to prioritize critical UAV traffic under congestion. Experimental results demonstrate that the proposed framework effectively balances resource conservation and delivery assurance. Compared to the static QoS policies present in the reference dataset, the proposed agents exhibit a superior ability to detect silent failures and to adaptively manage congestion, validating the effective- ness of intelligent, data-driven control policies for securing and optimizing critical UAV communications in unpredictable network environments.