论文标题
关于半监督学习中校准的重要性
On the Importance of Calibration in Semi-supervised Learning
论文作者
论文摘要
最先进的（SOTA）半监督学习（SSL）方法通过结合一致性正则化和伪标记的技术来利用标签和未标记数据的混合。在伪标记期间，该模型对未标记数据的预测用于训练，因此，模型校准对于缓解确认偏差很重要。然而，许多SOTA方法都针对模型性能进行了优化，而很少有针对性的焦点来改善模型校准。在这项工作中，我们从经验上证明，模型校准与模型性能密切相关，并建议通过近似贝叶斯技术改善校准。我们介绍了一个新的SSL模型系列，该系列可以优化校准并证明其在CIFAR-10，CIFAR-100和Imagenet的标准视觉基准中的有效性，可提高测试准确性15.9％。此外，我们还展示了它们在其他现实和具有挑战性的问题中的有效性，例如类不平衡的数据集和光子学科学。
State-of-the-art (SOTA) semi-supervised learning (SSL) methods have been highly successful in leveraging a mix of labeled and unlabeled data by combining techniques of consistency regularization and pseudo-labeling. During pseudo-labeling, the model's predictions on unlabeled data are used for training and thus, model calibration is important in mitigating confirmation bias. Yet, many SOTA methods are optimized for model performance, with little focus directed to improve model calibration. In this work, we empirically demonstrate that model calibration is strongly correlated with model performance and propose to improve calibration via approximate Bayesian techniques. We introduce a family of new SSL models that optimizes for calibration and demonstrate their effectiveness across standard vision benchmarks of CIFAR-10, CIFAR-100 and ImageNet, giving up to 15.9% improvement in test accuracy. Furthermore, we also demonstrate their effectiveness in additional realistic and challenging problems, such as class-imbalanced datasets and in photonics science.
