论文标题
最好的Bert和Crosloengual Bert：在多语言模型中更少
FinEst BERT and CroSloEngual BERT: less is more in multilingual models
论文作者
论文摘要
经过验证的蒙面语言模型已成为许多NLP问题的最新解决方案。但是，这项研究主要集中在英语上。尽管存在大规模的多语言模型，但研究表明单语模型会产生更好的结果。我们训练两个三语伯特式模型，一种用于芬兰语，爱沙尼亚语和英语，另一个用于克罗地亚，斯洛文尼亚语和英语。我们使用多语言BERT和XLM-R作为基础来评估他们在几个下游任务，NER，POS-TAGGING和依赖性解析的表现。新创建的最好的Bert和Crosloengual Bert在大多数单语和跨语言情况下都改善了所有任务的结果
Large pretrained masked language models have become state-of-the-art solutions for many NLP problems. The research has been mostly focused on English language, though. While massively multilingual models exist, studies have shown that monolingual models produce much better results. We train two trilingual BERT-like models, one for Finnish, Estonian, and English, the other for Croatian, Slovenian, and English. We evaluate their performance on several downstream tasks, NER, POS-tagging, and dependency parsing, using the multilingual BERT and XLM-R as baselines. The newly created FinEst BERT and CroSloEngual BERT improve the results on all tasks in most monolingual and cross-lingual situations
