论文标题
Meta-semi：半监督学习的元学习方法
Meta-Semi: A Meta-learning Approach for Semi-supervised Learning
论文作者
论文摘要
基于深度学习的半监督学习（SSL）算法在近年来导致了令人鼓舞的结果。但是，它们倾向于引入多个可调的超参数，从而使它们在实际的SSL方案中较不可能，因为标记的数据稀缺，无法进行广泛的超参数搜索。在本文中，我们提出了一种新型的基于元学习的SSL算法（Meta-semi），该算法仅需与标准监督深度学习算法相比，仅需调整一个额外的超参数，以在SSL的各种条件下实现竞争性能。我们首先定义一个元优化问题，该问题通过动态重新重新加权未标记样品的损失来最大程度地减少标记数据的损失，这些样品与训练过程中与软伪标签有关。由于元问题在计算上是直接求解的，因此我们提出了一种有效的算法来动态获得近似解决方案。从理论上讲，我们表明，在轻度条件下，元间会收敛到损失函数的固定点。从经验上讲，在具有挑战性的半监督CIFAR-100和STL-10任务上，Meta-Semi的最先进SSL算法极大地超过了SSL算法，并且在CIFAR-10和SVHN上取得了竞争性能。
Deep learning based semi-supervised learning (SSL) algorithms have led to promising results in recent years. However, they tend to introduce multiple tunable hyper-parameters, making them less practical in real SSL scenarios where the labeled data is scarce for extensive hyper-parameter search. In this paper, we propose a novel meta-learning based SSL algorithm (Meta-Semi) that requires tuning only one additional hyper-parameter, compared with a standard supervised deep learning algorithm, to achieve competitive performance under various conditions of SSL. We start by defining a meta optimization problem that minimizes the loss on labeled data through dynamically reweighting the loss on unlabeled samples, which are associated with soft pseudo labels during training. As the meta problem is computationally intensive to solve directly, we propose an efficient algorithm to dynamically obtain the approximate solutions. We show theoretically that Meta-Semi converges to the stationary point of the loss function on labeled data under mild conditions. Empirically, Meta-Semi outperforms state-of-the-art SSL algorithms significantly on the challenging semi-supervised CIFAR-100 and STL-10 tasks, and achieves competitive performance on CIFAR-10 and SVHN.
