论文标题
Winograd模式挑战的失败
The Defeat of the Winograd Schema Challenge
论文作者
论文摘要
Winograd模式挑战赛 - 一组涉及代词参考歧义的双句子似乎需要使用常识性知识 - 由Hector Levesque于2011年提出。到2019年，基于大型预培训的基于训练的变压器的语言模型和对这些问题进行精细调整，比90％的精确度更高。在本文中，我们回顾了Winograd模式挑战的历史，并讨论了过去十年来WSC上发生的一系列研究的持久贡献。我们讨论了为WSC开发的各种数据集的重要性，以及研究界对替代任务在评估AI系统智能中的作用的更深入的了解。
The Winograd Schema Challenge - a set of twin sentences involving pronoun reference disambiguation that seem to require the use of commonsense knowledge - was proposed by Hector Levesque in 2011. By 2019, a number of AI systems, based on large pre-trained transformer-based language models and fine-tuned on these kinds of problems, achieved better than 90% accuracy. In this paper, we review the history of the Winograd Schema Challenge and discuss the lasting contributions of the flurry of research that has taken place on the WSC in the last decade. We discuss the significance of various datasets developed for WSC, and the research community's deeper understanding of the role of surrogate tasks in assessing the intelligence of an AI system.
