论文标题
结构意识的句子相似性与递归最佳运输
Structural-Aware Sentence Similarity with Recursive Optimal Transport
论文作者
论文摘要
测量句子相似性是自然语言处理中的经典主题。即使深度学习模型在许多其他任务中取得了成功，轻度加权的相似性仍然具有特殊的实际意义。已经证明，一些具有更多理论见解的轻度加权相似之处比有监督的深度学习方法更强大。但是，成功的轻加权模型，例如单词移动器的距离[Kusner等，2015]或平滑的逆频率[Arora等，2017]未能检测到句子结构（即单词顺序）与句子结构的差异。为了解决这个问题，我们提出了递归的最佳运输（ROT）框架，以将结构信息与经典OT结合在一起。此外，我们进一步为句子提供了递归的最佳相似性（ROT），该句子具有从余弦的平均值与单词矢量的加权平均值与最佳传输之间的连接中的有价值的语义见解。与最佳运输相比，ROTS是结构感知的，并且具有较低的时间复杂性。我们超过20个句子纹理相似性（STS）数据集的实验显示了腐烂的明显优势，而不是所有弱监督的方法。详细的消融研究证明了腐烂和语义见解的有效性。
Measuring sentence similarity is a classic topic in natural language processing. Light-weighted similarities are still of particular practical significance even when deep learning models have succeeded in many other tasks. Some light-weighted similarities with more theoretical insights have been demonstrated to be even stronger than supervised deep learning approaches. However, the successful light-weighted models such as Word Mover's Distance [Kusner et al., 2015] or Smooth Inverse Frequency [Arora et al., 2017] failed to detect the difference from the structure of sentences, i.e. order of words. To address this issue, we present Recursive Optimal Transport (ROT) framework to incorporate the structural information with the classic OT. Moreover, we further develop Recursive Optimal Similarity (ROTS) for sentences with the valuable semantic insights from the connections between cosine similarity of weighted average of word vectors and optimal transport. ROTS is structural-aware and with low time complexity compared to optimal transport. Our experiments over 20 sentence textural similarity (STS) datasets show the clear advantage of ROTS over all weakly supervised approaches. Detailed ablation study demonstrate the effectiveness of ROT and the semantic insights.
