Sim-to-Real Learning-Based Nonlinear MPC for UAV Navigation and Collision Avoidance in Unknown Cluttered Environments

Quadrotor control in dynamic and unstructured environments requires robust and adaptive solutions to achieve agile and autonomous navigation. Nonlinear Model Predictive Control (NMPC) offers precise control through a nonlinear dynamical system model and online optimization over a short prediction horizon. However, its adaptability can be significantly enhanced through reinforcement learning. This study proposes a multimodal NMPC (MP-NMPC) policy, leveraging deep reinforcement learning to improve online optimization. Conditioned on the quadrotor’s local observations, the trained reinforcement learning policy dynamically selects adaptation parameters and feedforward control commands for the low-level NMPC controller. The search for adaptation variables and feedforward control signals is formulated as a Markov Decision Process (MDP), enabling reinforcement learning to address this optimization challenge. A multimodal deep reinforcement learning policy that combines depth images with inertial data is trained in simulation and transferred to a real quadrotor in a zero-shot fashion, facilitating online hyperparameter adaptation in highly dynamic unstructured environments. The significance of this approach is demonstrated through its application to challenging tasks such as agile navigation and obstacle avoidance, where it enables autonomous flight through complex environments and outperforms standard reinforcement learning methods.