论文标题
在Skipgram Word嵌入模型上具有负抽样：统一框架和噪声分布的影响
On SkipGram Word Embedding Models with Negative Sampling: Unified Framework and Impact of Noise Distributions
论文作者
论文摘要
Skipgram Word嵌入模型具有负抽样，或者简而言之，是一个优雅的单词嵌入模型系列。在本文中，我们为单词嵌入（称为文字上下文分类（WCC））制定了一个框架，该框架将SGN推广到广泛的模型家族。利用一些“噪声示例”的框架是通过理论分析证明的。通过实验研究了噪声分布对WCC嵌入模型学习的影响，这表明最好的噪声分布实际上是数据分布，就嵌入性能和训练过程中的收敛速度而言。在途中，我们发现了几种新颖的嵌入模型，这些模型胜过现有的WCC模型。
SkipGram word embedding models with negative sampling, or SGN in short, is an elegant family of word embedding models. In this paper, we formulate a framework for word embedding, referred to as Word-Context Classification (WCC), that generalizes SGN to a wide family of models. The framework, utilizing some "noise examples", is justified through a theoretical analysis. The impact of noise distribution on the learning of the WCC embedding models is studied experimentally, suggesting that the best noise distribution is in fact the data distribution, in terms of both the embedding performance and the speed of convergence during training. Along our way, we discover several novel embedding models that outperform the existing WCC models.
