论文标题
验证语言模型中的科学和创造性类比
Scientific and Creative Analogies in Pretrained Language Models
论文作者
论文摘要
本文研究了大规模预处理的语言模型（例如BERT和GPT-2）中类比的编码。现有的类比数据集通常集中在有限的类似关系上，与类比之间的两个领域相似。作为一个更现实的设置，我们介绍了科学和创造性的类比法数据集（SCAN），这是一个新颖的类比数据集，该数据集包含对跨不同领域的多个属性和关系结构的系统映射。使用此数据集，我们测试了几种广泛使用的审前语言模型（LMS）的类似推理能力。我们发现，最先进的LMS在这些复杂的类比任务上实现了较低的性能，突出了类比理解所带来的挑战。
This paper examines the encoding of analogy in large-scale pretrained language models, such as BERT and GPT-2. Existing analogy datasets typically focus on a limited set of analogical relations, with a high similarity of the two domains between which the analogy holds. As a more realistic setup, we introduce the Scientific and Creative Analogy dataset (SCAN), a novel analogy dataset containing systematic mappings of multiple attributes and relational structures across dissimilar domains. Using this dataset, we test the analogical reasoning capabilities of several widely-used pretrained language models (LMs). We find that state-of-the-art LMs achieve low performance on these complex analogy tasks, highlighting the challenges still posed by analogy understanding.
