论文标题
语言模型学习位置角色映射吗？
Do Language Models Learn Position-Role Mappings?
论文作者
论文摘要
自然语言中对位置角色映射的知识如何学到？我们在计算环境中探讨了这个问题，测试了各种表现出色的语言模型（Bert，Roberta和Distilbert）是否表现出对这些映射的知识，以及这些知识是否持续到句法，结构和词汇交替的交替。在实验1中，我们表明这些神经模型确实确实识别了主题和受体在双授予构造中的区别，并且这些不同的模式在构造类型中共享。我们在实验2中加强了这一发现，表明在一个范式中对新主题和类似于接受者的代币进行微调模型，使模型可以正确预测其在其他范式中的位置，这表明这些映射的知识是共享的，而不是独立学习的。但是，当任务涉及具有新型坦率动词的构造时，我们确实会观察到这种概括的某些局限性，并暗示了一种依赖模型性能的词汇特异性程度。
How is knowledge of position-role mappings in natural language learned? We explore this question in a computational setting, testing whether a variety of well-performing pertained language models (BERT, RoBERTa, and DistilBERT) exhibit knowledge of these mappings, and whether this knowledge persists across alternations in syntactic, structural, and lexical alternations. In Experiment 1, we show that these neural models do indeed recognize distinctions between theme and recipient roles in ditransitive constructions, and that these distinct patterns are shared across construction type. We strengthen this finding in Experiment 2 by showing that fine-tuning these language models on novel theme- and recipient-like tokens in one paradigm allows the models to make correct predictions about their placement in other paradigms, suggesting that the knowledge of these mappings is shared rather than independently learned. We do, however, observe some limitations of this generalization when tasks involve constructions with novel ditransitive verbs, hinting at a degree of lexical specificity which underlies model performance.
