论文标题
注意硬审判
Pay Attention to Hard Trials
论文作者
论文摘要
在测试试验中评估了说话者识别系统的性能。尽管与裁缝的统治者一样至关重要，但迄今尚未对试验进行仔细的治疗，并且大多数现有的基准测试均通过幼稚的交叉配对构成试验。在本文中，我们认为交叉生产方法会产生压倒性的易于试验，进而导致系统和技术比较的潜在偏见。为了解决这个问题，我们提倡更多地关注硬试验。我们提出了一种基于SVM的方法来识别硬试验，并使用它来为Voxceleb1和Sitw构建新的评估集。借助新集合，我们可以重新评估一些最近的技术的贡献。代码和确定的硬试验将在线发布http://project.cslt.org。
Performance of speaker recognition systems is evaluated on test trials. Although as crucial as rulers for tailors, trials have not been carefully treated so far, and most existing benchmarks compose trials by naive cross-pairing. In this paper, we argue that the cross-pairing approach produces overwhelming easy trials, which in turn leads to potential bias in system and technique comparison. To solve the problem, we advocate more attention to hard trials. We present an SVM-based approach to identifying hard trials and use it to construct new evaluation sets for VoxCeleb1 and SITW. With the new sets, we can re-evaluate the contribution of some recent technologies. The code and the identified hard trials will be published online at http://project.cslt.org.
