论文标题
Advantagenas：有效的神经体系结构搜索和信用分配
AdvantageNAS: Efficient Neural Architecture Search with Credit Assignment
论文作者
论文摘要
神经体系结构搜索（NAS）是一种无需人类努力或专业知识而自动设计神经网络体系结构的方法。但是，NAS的高计算成本限制了其在商业应用中的使用。最近的两个NAS范式，即单发和稀疏的繁殖，分别降低了时间和空间复杂性，为解决此问题提供了线索。在本文中，我们提出了一种新颖的搜索策略，以实现单发和稀疏的繁殖NAS，即Advantagenas，该策略通过减少搜索迭代的数量来进一步降低NAS的时间复杂性。 Advantagenas是一种基于梯度的方法，可以通过在架构更新梯度估算中引入信用分配来提高搜索效率。 NAS-Bench-201和PTB数据集的实验表明，与现有的稀疏繁殖NAS相比，Advantagenas在有限时间预算下的性能较高。为了进一步揭示Advantagenas的可靠性，我们从理论上进行了调查，并发现它可以单调地改善预期损失，从而融合。
Neural architecture search (NAS) is an approach for automatically designing a neural network architecture without human effort or expert knowledge. However, the high computational cost of NAS limits its use in commercial applications. Two recent NAS paradigms, namely one-shot and sparse propagation, which reduce the time and space complexities, respectively, provide clues for solving this problem. In this paper, we propose a novel search strategy for one-shot and sparse propagation NAS, namely AdvantageNAS, which further reduces the time complexity of NAS by reducing the number of search iterations. AdvantageNAS is a gradient-based approach that improves the search efficiency by introducing credit assignment in gradient estimation for architecture updates. Experiments on the NAS-Bench-201 and PTB dataset show that AdvantageNAS discovers an architecture with higher performance under a limited time budget compared to existing sparse propagation NAS. To further reveal the reliabilities of AdvantageNAS, we investigate it theoretically and find that it monotonically improves the expected loss and thus converges.
