论文标题
感知世界:基于文本游戏的问题指导的加强学习
Perceiving the World: Question-guided Reinforcement Learning for Text-based Games
论文作者
论文摘要
基于文本的游戏提供了一种研究自然语言处理的交互式方式。尽管深度强化学习已经显示出在开发游戏玩具代理方面的有效性,但较低的样本效率和较大的动作空间仍然是阻碍DRL应用于现实世界的两个主要挑战。在本文中,我们通过引入世界感知的模块来应对挑战,该模块通过回答有关环境的问题自动分解任务和修剪行动。然后,我们提出了一个两阶段的培训框架,以从强化学习中解除语言学习,从而进一步提高了样本效率。实验结果表明,所提出的方法显着提高了性能和样本效率。此外,它显示出针对复合误差和有限的预训练数据的鲁棒性。
Text-based games provide an interactive way to study natural language processing. While deep reinforcement learning has shown effectiveness in developing the game playing agent, the low sample efficiency and the large action space remain to be the two major challenges that hinder the DRL from being applied in the real world. In this paper, we address the challenges by introducing world-perceiving modules, which automatically decompose tasks and prune actions by answering questions about the environment. We then propose a two-phase training framework to decouple language learning from reinforcement learning, which further improves the sample efficiency. The experimental results show that the proposed method significantly improves the performance and sample efficiency. Besides, it shows robustness against compound error and limited pre-training data.