RL BASED UAV OBSTACLE AVOIDANCE USING LOW COST AND LOW POWER TIME OF FLIGHT SENSORS

Autonomous navigation of micro aerial vehicles in cluttered indoor environments remains a challenging problem due to the need for accurate control, robustness against aerodynamic disturbances, and effective obstacle avoidance under severe sensing and computational constraints. This thesis presents a hierarchical reinforcement learning framework for autonomous navigation of a micro aerial vehicle, developed and validated in NVIDIA Isaac Lab and subsequently transferred to and experimentally evaluated on a real platform. The proposed control architecture is organised into two hierarchical levels. A pre-trained low-level velocity policy, kept frozen during the subsequent training phase, is responsible for tracking body-frame velocity commands, while a high-level policy, trained through Proximal Policy Optimization (PPO) at 25 Hz, generates these references to achieve position and yaw waypoint tracking. The training process is carried out in massively parallel domain-randomised environments, including asymmetric six-wall indoor scenarios, randomised waypoint configurations, and a wind disturbance model combining mean airflow, turbulence, and stochastic gusts with intensities comparable to the vehicle weight. Since external disturbances are not directly observable, compensation is achieved through a distance-gated leaky integrator of the tracking error, providing an implicit disturbance estimation mechanism. To enable obstacle avoidance and facilitate sim-to-real transfer, four STMicroelectronics VL53L9CX time-of-flight sensors are modelled in simulation through GPU-based ray casting, reproducing the sensor field of view and acquisition modes described in the datasheet. The resulting depth information is processed through geometric min-pooling into angular sectors, producing a compact observation representation suitable for implementation within the ROS 2 architecture of the real flight platform. A comprehensive evaluation framework is introduced to analyse navigation performance, including steady-state positioning accuracy, overshoot, command frequency content, integrator behaviour, and obstacle clearance with respect to a sensor-independent geometric ground truth. Experimental validation, conducted both in simulation and on the real platform, demonstrates centimetre-level steady-state positioning accuracy and highlights the conservativeness of the perception-based collision criterion. Furthermore, the analysis identifies the limited angular coverage of a four-sensor configuration as a fundamental constraint for reliable obstacle avoidance in complex indoor environments.