论文标题
通过双向变形金刚在顺序决策问题上柔韧性推断
Towards Flexible Inference in Sequential Decision Problems via Bidirectional Transformers
论文作者
论文摘要
随机掩盖和预测单词令牌是针对各种下游任务的预训练语言模型的成功方法。在这项工作中，我们观察到，同一想法也自然地适用于顺序决策，在这些决策中，许多经过精心训练的任务，例如行为克隆，离线RL，逆动力学和WayPoint条件，对应于一系列状态，动作和返回的不同序列掩码。我们介绍了Flexibit框架，该框架提供了一种统一的方式来指定可以在许多不同的顺序决策任务上培训的模型。我们表明，单个Flexibit模型同时能够执行许多与专用模型相似或更好的性能的任务。此外，我们表明，通过对特定的感兴趣任务进行微调，可以通过微调我们的一般模型来进一步提高绩效。
Randomly masking and predicting word tokens has been a successful approach in pre-training language models for a variety of downstream tasks. In this work, we observe that the same idea also applies naturally to sequential decision making, where many well-studied tasks like behavior cloning, offline RL, inverse dynamics, and waypoint conditioning correspond to different sequence maskings over a sequence of states, actions, and returns. We introduce the FlexiBiT framework, which provides a unified way to specify models which can be trained on many different sequential decision making tasks. We show that a single FlexiBiT model is simultaneously capable of carrying out many tasks with performance similar to or better than specialized models. Additionally, we show that performance can be further improved by fine-tuning our general model on specific tasks of interest.
