Deep Reinforcement Learning for Autonomous Map-Less Navigation of a Flying Robot

Flying robots are expected to be used in many tasks, such as aerial delivery, inspection inside dangerous areas, and rescue. However, their deployment in unstructured and highly dynamic environments has been limited. This paper proposes a novel approach for enabling a micro-aerial vehicle (MAV) system equipped with a laser rangefinder and depth sensor to autonomously navigate and explore an unknown indoor or outdoor environment. We built a modular deep-Q-network architecture to fuse information from multiple sensors mounted onboard a vehicle. The developed algorithm can perform collision-free flights in the real world, while being trained entirely on a 3D simulator. The proposed method does not require prior expert demonstrations, 3D mapping, or path planning. It transforms fused sensory data into a velocity control input for a robot through an end-to-end convolutional neural network (CNN). The obtained policy was compared to a simulation using the conventional potential-field method. Our approach achieves zero-shot transfer from simulation to real-world environments that were never experienced during training by simulating realistic sensor data. Several intensive experiments were conducted to demonstrate the effectiveness of our system for safely flying in dynamic outdoor and indoor environments. The supplementary videos for the actual flight tests can be accessed at https://bit.ly/2SEw8dQ.