论文标题
Blackbird的语言矩阵（BLMS）：研究神经网络中分离概括的新基准
Blackbird's language matrices (BLMs): a new benchmark to investigate disentangled generalisation in neural networks
论文作者
论文摘要
机器学习体系结构的当前成功基于计算上昂贵的算法和大量数据。我们需要开发任务和数据来培训网络，以达到更复杂和更具组成的技能。在本文中，我们说明了Blackbird的语言矩阵（BLMS），这是一种新型的语法数据集，开发了用于测试Raven渐进式矩阵的语言变体，这是一种通常基于视觉刺激的智能测试。该数据集由44800个句子组成，该句子通常是为了支持对当前模型的语法协议规则的语言掌握及其概括的能力的研究。我们介绍数据集的逻辑，即大规模自动构建数据的方法以及学习它们的架构。通过错误分析和有关数据集变化的几个实验，我们证明了该语言任务以及实例化的数据提供了一个新的具有挑战性的测试床，以了解概括和抽象。
Current successes of machine learning architectures are based on computationally expensive algorithms and prohibitively large amounts of data. We need to develop tasks and data to train networks to reach more complex and more compositional skills. In this paper, we illustrate Blackbird's language matrices (BLMs), a novel grammatical dataset developed to test a linguistic variant of Raven's progressive matrices, an intelligence test usually based on visual stimuli. The dataset consists of 44800 sentences, generatively constructed to support investigations of current models' linguistic mastery of grammatical agreement rules and their ability to generalise them. We present the logic of the dataset, the method to automatically construct data on a large scale and the architecture to learn them. Through error analysis and several experiments on variations of the dataset, we demonstrate that this language task and the data that instantiate it provide a new challenging testbed to understand generalisation and abstraction.
