论文标题
RETRO-RL:使用深度强化学习的倾斜旋转无人机增强名义控制器
Retro-RL: Reinforcing Nominal Controller With Deep Reinforcement Learning for Tilting-Rotor Drones
论文作者
论文摘要
将无人机应用扩展到复杂任务的研究需要稳定的控制框架。最近,在许多研究中,对机器人控制以完成复杂的任务已经利用了深入的增强学习(RL)算法。不幸的是,由于难以解释学习的政策和缺乏稳定性保证,尤其是对于诸如越来越多的无人机,尤其是诸如越来越多的稳定性的任务,深层RL算法可能不适合直接部署到现实世界的机器人平台中。本文提出了一种新型的混合体系结构,该结构通过使用无模型的深入RL算法学习的强大策略来增强名义控制器。所提出的架构采用不确定性感受的控制混合器来保留名义控制器的保证稳定性,同时使用了学习策略的扩展性能。该政策在模拟环境中进行了数千个域随机化的培训,以实现多样化的不确定性绩效。通过现实世界实验验证了所提出的方法的性能,然后与传统的控制器和经过香草深RL算法训练的基于最新的学习控制器进行了比较。
Studies that broaden drone applications into complex tasks require a stable control framework. Recently, deep reinforcement learning (RL) algorithms have been exploited in many studies for robot control to accomplish complex tasks. Unfortunately, deep RL algorithms might not be suitable for being deployed directly into a real-world robot platform due to the difficulty in interpreting the learned policy and lack of stability guarantee, especially for a complex task such as a wall-climbing drone. This paper proposes a novel hybrid architecture that reinforces a nominal controller with a robust policy learned using a model-free deep RL algorithm. The proposed architecture employs an uncertainty-aware control mixer to preserve guaranteed stability of a nominal controller while using the extended robust performance of the learned policy. The policy is trained in a simulated environment with thousands of domain randomizations to achieve robust performance over diverse uncertainties. The performance of the proposed method was verified through real-world experiments and then compared with a conventional controller and the state-of-the-art learning-based controller trained with a vanilla deep RL algorithm.