论文标题
通过句法协议测试评估德国变压器语言模型
Evaluating German Transformer Language Models with Syntactic Agreement Tests
论文作者
论文摘要
预先训练的变压器语言模型（TLMS）最近重新塑造了自然语言处理（NLP）：大多数最新的NLP模型现在都在TLMS之上运行，以受益于上下文化和知识诱导。为了解释他们的成功，科学界进行了许多分析。除其他方法外，还利用了句法协议测试来分析TLM。但是，大多数研究都是针对英语进行的。在这项工作中，我们分析了德国TLM。为此，我们设计了许多协议任务，其中一些任务考虑了德语的特殊性。我们的实验结果表明，最先进的德国TLM通常在协议任务上表现良好，但我们还确定并讨论将其限制的句法结构。
Pre-trained transformer language models (TLMs) have recently refashioned natural language processing (NLP): Most state-of-the-art NLP models now operate on top of TLMs to benefit from contextualization and knowledge induction. To explain their success, the scientific community conducted numerous analyses. Besides other methods, syntactic agreement tests were utilized to analyse TLMs. Most of the studies were conducted for the English language, however. In this work, we analyse German TLMs. To this end, we design numerous agreement tasks, some of which consider peculiarities of the German language. Our experimental results show that state-of-the-art German TLMs generally perform well on agreement tasks, but we also identify and discuss syntactic structures that push them to their limits.
