论文标题
部分可观测时空混沌系统的无模型预测
Differentiable Parsing and Visual Grounding of Natural Language Instructions for Object Placement
论文作者
论文摘要
我们提出了一种新方法，解析和视觉接地（Paragon），用于在对象放置任务中接地自然语言。自然语言通常描述了对象和空间关系，具有组成性和歧义性，这是有效语言基础的两个主要障碍。对于组成性，Paragon将语言指令解析为以对象为中心的图表表示，以单独地将地面对象。为了歧义，Paragon使用一种新型的基于粒子的图形神经网络来理解不确定性的对象放置。本质上，Paragon将解析算法集成到一个概率，数据驱动的学习框架中。它与与复杂，模棱两可的语言输入的鲁棒性有关的数据是完全可区分和训练的端到端。
We present a new method, PARsing And visual GrOuNding (ParaGon), for grounding natural language in object placement tasks. Natural language generally describes objects and spatial relations with compositionality and ambiguity, two major obstacles to effective language grounding. For compositionality, ParaGon parses a language instruction into an object-centric graph representation to ground objects individually. For ambiguity, ParaGon uses a novel particle-based graph neural network to reason about object placements with uncertainty. Essentially, ParaGon integrates a parsing algorithm into a probabilistic, data-driven learning framework. It is fully differentiable and trained end-to-end from data for robustness against complex, ambiguous language input.
