论文标题
Relativenas：通过缓慢学习的相对神经体系结构搜索
RelativeNAS: Relative Neural Architecture Search via Slow-Fast Learning
论文作者
论文摘要
尽管卷积神经网络（CNN）在计算机视觉中取得了显着的成功，但手动设计CNN还是耗时且容易出错的。在各种神经体系结构搜索（NAS）方法中，他们有动力自动化高性能CNN的设计，由于其独特的角色，可区分的NAS和基于人群的NAS正在吸引越来越多的兴趣。为了从优点受益，同时克服了两者的缺陷，这项工作提出了一种新型的NAS方法Relativenas。作为有效搜索的关键，Relativenas以成对的方式在快速学习者（即具有相对较高精度的网络）和慢学习者之间执行关节学习。此外，由于Relativenas仅需要低保真绩效估算来区分每对快速学习者和慢速学习者，因此它可以节省某些计算成本，以训练候选体系结构。拟议的Relativenas带来了几个独特的优势：（1）它在ImageNet上实现了最先进的性能，前1位错误率为24.88％，即表现优于Darts和Amoebanet-B分别为1.82％和1.12％； （2）它仅在一个1080TI GPU上花费9个小时才能获得发现的细胞，即分别比飞镖和Amoebanet快3.75倍和7875倍。 （3）它规定，在CIFAR-10上获得的发现的细胞可以直接转移到对象检测，语义分割和关键点检测中，从而在Pascal VOC上获得73.1％MAP的竞争结果，在CityScapes上获得78.7％MIOU，分别在MSCOCO上获得68.5％的AP。 Relativenas的实现可在https://github.com/emi-group/relativenas上获得
Despite the remarkable successes of Convolutional Neural Networks (CNNs) in computer vision, it is time-consuming and error-prone to manually design a CNN. Among various Neural Architecture Search (NAS) methods that are motivated to automate designs of high-performance CNNs, the differentiable NAS and population-based NAS are attracting increasing interests due to their unique characters. To benefit from the merits while overcoming the deficiencies of both, this work proposes a novel NAS method, RelativeNAS. As the key to efficient search, RelativeNAS performs joint learning between fast-learners (i.e. networks with relatively higher accuracy) and slow-learners in a pairwise manner. Moreover, since RelativeNAS only requires low-fidelity performance estimation to distinguish each pair of fast-learner and slow-learner, it saves certain computation costs for training the candidate architectures. The proposed RelativeNAS brings several unique advantages: (1) it achieves state-of-the-art performance on ImageNet with top-1 error rate of 24.88%, i.e. outperforming DARTS and AmoebaNet-B by 1.82% and 1.12% respectively; (2) it spends only nine hours with a single 1080Ti GPU to obtain the discovered cells, i.e. 3.75x and 7875x faster than DARTS and AmoebaNet respectively; (3) it provides that the discovered cells obtained on CIFAR-10 can be directly transferred to object detection, semantic segmentation, and keypoint detection, yielding competitive results of 73.1% mAP on PASCAL VOC, 78.7% mIoU on Cityscapes, and 68.5% AP on MSCOCO, respectively. The implementation of RelativeNAS is available at https://github.com/EMI-Group/RelativeNAS
