论文标题
无监督的词汇取代，脱皮的嵌入
Unsupervised Lexical Substitution with Decontextualised Embeddings
论文作者
论文摘要
我们提出了一种使用预训练的语言模型的新的无监督方法，用于词汇替代。与以前使用语言模型的生成能力预测替代品的方法相比，我们的方法基于上下文化和脱皮的单词嵌入的相似性检索替代品，即在多个上下文中单词的平均上下文表示。我们在英语和意大利语中进行实验，并表明我们的方法基本上要优于强大的基准，并在没有任何明确的监督或微调的情况下建立了新的最先进的基础。我们进一步表明，我们的方法在预测低频替代物方面的表现特别出色，并且还产生了多种替代候选者列表，从而减少了根据文章 - 名称协议引起的形态寄电或形态句法偏见。
We propose a new unsupervised method for lexical substitution using pre-trained language models. Compared to previous approaches that use the generative capability of language models to predict substitutes, our method retrieves substitutes based on the similarity of contextualised and decontextualised word embeddings, i.e. the average contextual representation of a word in multiple contexts. We conduct experiments in English and Italian, and show that our method substantially outperforms strong baselines and establishes a new state-of-the-art without any explicit supervision or fine-tuning. We further show that our method performs particularly well at predicting low-frequency substitutes, and also generates a diverse list of substitute candidates, reducing morphophonetic or morphosyntactic biases induced by article-noun agreement.
