论文标题
使用独立的干扰评估和改善多转反应生成的上下文注意力分布
Evaluating and Improving Context Attention Distribution on Multi-Turn Response Generation using Self-Contained Distractions
论文作者
论文摘要
尽管基于开放域的对话代理的进展迅速，但大多数部署的系统将对话环境视为单转，而处理多转移环境的系统的研究较少。缺乏可靠的度量标准来评估多转弯建模以及改进其的有效解决方案。在本文中，我们关注基于多转变的对话代理的基本组成部分：上下文注意分布，即系统如何在对话的环境中分发他们的注意力。为了评估该组件，我们引入了一种新型的基于注意力机智的度量：DAS比率。为了提高该组件的绩效，我们提出了一种采用独立干扰的优化策略。我们在Ubuntu Chatlogs数据集上进行的实验表明，具有可比性的模型可以通过其在上下文注意力分布上的能力来区分。我们提出的优化策略将拟议指标上的非层次结构和分层模型提高了大约10％的基线。
Despite the rapid progress of open-domain generation-based conversational agents, most deployed systems treat dialogue contexts as single-turns, while systems dealing with multi-turn contexts are less studied. There is a lack of a reliable metric for evaluating multi-turn modelling, as well as an effective solution for improving it. In this paper, we focus on an essential component of multi-turn generation-based conversational agents: context attention distribution, i.e. how systems distribute their attention on dialogue's context. For evaluation of this component, We introduce a novel attention-mechanism-based metric: DAS ratio. To improve performance on this component, we propose an optimization strategy that employs self-contained distractions. Our experiments on the Ubuntu chatlogs dataset show that models with comparable perplexity can be distinguished by their ability on context attention distribution. Our proposed optimization strategy improves both non-hierarchical and hierarchical models on the proposed metric by about 10% from baselines.
