论文标题
无单元的潜在探索
Cell-Free Latent Go-Explore
论文作者
论文摘要
在本文中，我们介绍了潜在的探索（LGE），这是一种基于探索加固学习（RL）探索的简单而通用的方法。最初引入了Go-explore，并具有强大的域知识约束，以将状态空间划分为单元。但是，在大多数实际情况下，从原始观察中汲取领域知识是复杂而乏味的。如果细胞分配不够有用，则可以完全无法探索环境。我们认为，可以通过利用学习的潜在表示，可以将Go-explore方法推广到任何环境，而无需细胞。因此，我们表明LGE可以灵活地与学习潜在表示的任何策略相结合。我们的结果表明，LGE虽然比Go-explore更简单，但在纯粹的探索（包括Montezuma的复仇）上的纯粹探索方面，更强大，并且优于最先进的算法。 LGE实现可在https://github.com/qgallouedec/lge上作为开源。
In this paper, we introduce Latent Go-Explore (LGE), a simple and general approach based on the Go-Explore paradigm for exploration in reinforcement learning (RL). Go-Explore was initially introduced with a strong domain knowledge constraint for partitioning the state space into cells. However, in most real-world scenarios, drawing domain knowledge from raw observations is complex and tedious. If the cell partitioning is not informative enough, Go-Explore can completely fail to explore the environment. We argue that the Go-Explore approach can be generalized to any environment without domain knowledge and without cells by exploiting a learned latent representation. Thus, we show that LGE can be flexibly combined with any strategy for learning a latent representation. Our results indicate that LGE, although simpler than Go-Explore, is more robust and outperforms state-of-the-art algorithms in terms of pure exploration on multiple hard-exploration environments including Montezuma's Revenge. The LGE implementation is available as open-source at https://github.com/qgallouedec/lge.
