论文标题
MAML和理论启发的神经体系结构的全球融合搜索几乎没有学习的学习
Global Convergence of MAML and Theory-Inspired Neural Architecture Search for Few-Shot Learning
论文作者
论文摘要
模型 - 敏捷的元学习（MAML）及其变体已成为几次学习的流行方法。然而，由于深神经网（DNN）的非跨性别性和MAML的双层公式，因此MAML具有DNN的理论特性仍然很大。在本文中，我们首先证明具有过度参数DNNS的MAML可以以线性速率收敛到全球Optima。我们的收敛分析表明，具有过度参数的DNN的MAML等同于内核回归，我们将其称为元神经切线核（Metantk）。然后，我们提出了一种新的无培训神经体系结构搜索（NAS）方法Metantk-Nas，用于使用Metantk进行排名和选择体系结构的几次学习。从经验上讲，我们将Metantk-NAS与以前的NAS方法进行了比较，该方法在两个流行的少量学习基准测试基准（Miniimagenet）和Tieredimagenet上进行了比较。我们表明，Metantk-NAS的性能比最先进的NAS方法可比性或更好，而在享受超过100倍的速度的同时，设计了几次学习。我们认为，Metantk-NAS的效率使其对许多现实世界任务更加实用。
Model-agnostic meta-learning (MAML) and its variants have become popular approaches for few-shot learning. However, due to the non-convexity of deep neural nets (DNNs) and the bi-level formulation of MAML, the theoretical properties of MAML with DNNs remain largely unknown. In this paper, we first prove that MAML with over-parameterized DNNs is guaranteed to converge to global optima at a linear rate. Our convergence analysis indicates that MAML with over-parameterized DNNs is equivalent to kernel regression with a novel class of kernels, which we name as Meta Neural Tangent Kernels (MetaNTK). Then, we propose MetaNTK-NAS, a new training-free neural architecture search (NAS) method for few-shot learning that uses MetaNTK to rank and select architectures. Empirically, we compare our MetaNTK-NAS with previous NAS methods on two popular few-shot learning benchmarks, miniImageNet, and tieredImageNet. We show that the performance of MetaNTK-NAS is comparable or better than the state-of-the-art NAS method designed for few-shot learning while enjoying more than 100x speedup. We believe the efficiency of MetaNTK-NAS makes itself more practical for many real-world tasks.
