论文标题
了解多语言复发性神经网络中的跨语性句法转移
Understanding Cross-Lingual Syntactic Transfer in Multilingual Recurrent Neural Networks
论文作者
论文摘要
现在已经确定，现代神经语言模型可以同时在多种语言上成功训练，而不会改变基础体系结构。但是，在这些模型中，在语言之间真正共享了什么样的知识？多语言训练是否主要导致词汇表示空间的一致性，还是还可以共享纯语法知识？在本文中，我们使用各种模型和探测任务来解剖不同形式的跨语性转移，并寻找其最确定的因素。我们发现，将LMS暴露于相关语言并不总是会增加目标语言中的语法知识，而词汇语义转移的最佳条件对于句法转移可能并不是最佳的。
It is now established that modern neural language models can be successfully trained on multiple languages simultaneously without changes to the underlying architecture. But what kind of knowledge is really shared among languages within these models? Does multilingual training mostly lead to an alignment of the lexical representation spaces or does it also enable the sharing of purely grammatical knowledge? In this paper we dissect different forms of cross-lingual transfer and look for its most determining factors, using a variety of models and probing tasks. We find that exposing our LMs to a related language does not always increase grammatical knowledge in the target language, and that optimal conditions for lexical-semantic transfer may not be optimal for syntactic transfer.
