论文标题
关于在非天然图像数据集中获胜门票的可转让性
On the Transferability of Winning Tickets in Non-Natural Image Datasets
论文作者
论文摘要
我们研究了修剪神经网络的概括特性，这些神经网络是自然图像数据集中彩票假设的赢家。我们在培训数据稀缺的情况下分析了它们的潜力，并且来自非天然领域。具体而言，我们研究了在流行的CIFAR-10/100和时尚驱动器数据集中发现的修剪模型，该模型是否概括为来自数字病理和数字遗产领域的七个不同数据集。我们的结果表明，转移和训练稀疏体系结构比较大的参数化模型都有显着好处，因为在我们所有的实验中，修剪过的网络，彩票票证假设的获奖者，都大大优于其较大的未经审查的对应物。这些结果表明，获胜的初始化确实包含了一定程度上仿制的归纳偏差，尽管根据我们在生物医学数据集上的实验报告，它们的概括性能比迄今为止在文献中观察到的更具限制性。
We study the generalization properties of pruned neural networks that are the winners of the lottery ticket hypothesis on datasets of natural images. We analyse their potential under conditions in which training data is scarce and comes from a non-natural domain. Specifically, we investigate whether pruned models that are found on the popular CIFAR-10/100 and Fashion-MNIST datasets, generalize to seven different datasets that come from the fields of digital pathology and digital heritage. Our results show that there are significant benefits in transferring and training sparse architectures over larger parametrized models, since in all of our experiments pruned networks, winners of the lottery ticket hypothesis, significantly outperform their larger unpruned counterparts. These results suggest that winning initializations do contain inductive biases that are generic to some extent, although, as reported by our experiments on the biomedical datasets, their generalization properties can be more limiting than what has been so far observed in the literature.
