论文标题
智能磁性微型机器人学会在深度加固学习中游泳
Smart Magnetic Microrobots Learn to Swim with Deep Reinforcement Learning
论文作者
论文摘要
游泳微型机器人越来越多地通过复杂的材料和动态形状开发，并有望在复杂的环境中运行，在复杂的环境中，系统动力学难以建模，并且对微型机器人的位置控制并不直接实现。深度强化学习是一种有前途的方法，它可以自主开发可创建智能微型机器人的强大控制器，这可以使其行为适应在未表征的环境中操作，而无需对系统动态进行建模。在这里，我们报告了一种智能的螺旋磁性水凝胶微型机器人的发展，该微型机器人使用软演员评论家增强学习算法来自主地得出控制策略，该策略允许微型机器人通过未传感器化的生物模拟环境游泳，该环境是在不同的磁场中控制了由三轴电磁阵列产生的磁场，该磁场是由电磁阵列带来的。强化学习者以少于100,000个培训步骤学习了成功的控制政策，这证明了快速学习的样本效率。我们还证明，我们可以通过将数学功能拟合到通过回归中的数学策略的行动分布中，微调增强学习者学到的控制政策。应用于微型机器人控制的深度加强学习可能会显着扩大下一代微型机器人的能力。
Swimming microrobots are increasingly developed with complex materials and dynamic shapes and are expected to operate in complex environments in which the system dynamics are difficult to model and positional control of the microrobot is not straightforward to achieve. Deep reinforcement learning is a promising method of autonomously developing robust controllers for creating smart microrobots, which can adapt their behavior to operate in uncharacterized environments without the need to model the system dynamics. Here, we report the development of a smart helical magnetic hydrogel microrobot that used the soft actor critic reinforcement learning algorithm to autonomously derive a control policy which allowed the microrobot to swim through an uncharacterized biomimetic fluidic environment under control of a time varying magnetic field generated from a three-axis array of electromagnets. The reinforcement learning agent learned successful control policies with fewer than 100,000 training steps, demonstrating sample efficiency for fast learning. We also demonstrate that we can fine tune the control policies learned by the reinforcement learning agent by fitting mathematical functions to the learned policy's action distribution via regression. Deep reinforcement learning applied to microrobot control is likely to significantly expand the capabilities of the next generation of microrobots.
