论文标题
语言嵌入会捕获量表吗？
Do Language Embeddings Capture Scales?
论文作者
论文摘要
预审前的语言模型（LMS）已被证明具有重要的语言，常识和事实知识。在这种情况下尚未研究的一种知识形式是有关对象标量幅度的信息。我们表明，预验证的语言模型捕获了大量此信息，但缺乏一般常识性推理所需的能力。我们将训练和算术中的上下文信息确定为影响其性能的两个关键因素，并表明一种简单的规范数字方法可以对结果产生重大影响。
Pretrained Language Models (LMs) have been shown to possess significant linguistic, common sense, and factual knowledge. One form of knowledge that has not been studied yet in this context is information about the scalar magnitudes of objects. We show that pretrained language models capture a significant amount of this information but are short of the capability required for general common-sense reasoning. We identify contextual information in pre-training and numeracy as two key factors affecting their performance and show that a simple method of canonicalizing numbers can have a significant effect on the results.
