论文标题
NISPA：稀疏网络中持续学习的神经启发的稳态适应性
NISPA: Neuro-Inspired Stability-Plasticity Adaptation for Continual Learning in Sparse Networks
论文作者
论文摘要
持续学习的目标（CL）是随着时间的流逝学习不同的任务。与CL相关的主要Desiderata是保持旧任务的绩效，利用后者来改善对未来任务的学习，并在培训过程中引入最小的开销（例如，不需要成长的模型或再培训）。我们建议通过固定密度的稀疏神经网络来解决这些避难所的稳定性适应性（NISPA）体系结构。 NISPA形成了稳定的途径，可以从较旧的任务中保留知识。此外，NISPA使用连接重新设计来创建新的塑料路径，以重用有关新任务的现有知识。我们对EMNIST，FashionMnist，CIFAR10和CIFAR100数据集的广泛评估表明，NISPA的表现明显优于代表性的最先进的持续学习基线，并且与盆地相比，它的可学习参数最多少了十倍。我们还认为稀疏是持续学习的重要组成部分。 NISPA代码可从https://github.com/burakgurbuz97/nispa获得。
The goal of continual learning (CL) is to learn different tasks over time. The main desiderata associated with CL are to maintain performance on older tasks, leverage the latter to improve learning of future tasks, and to introduce minimal overhead in the training process (for instance, to not require a growing model or retraining). We propose the Neuro-Inspired Stability-Plasticity Adaptation (NISPA) architecture that addresses these desiderata through a sparse neural network with fixed density. NISPA forms stable paths to preserve learned knowledge from older tasks. Also, NISPA uses connection rewiring to create new plastic paths that reuse existing knowledge on novel tasks. Our extensive evaluation on EMNIST, FashionMNIST, CIFAR10, and CIFAR100 datasets shows that NISPA significantly outperforms representative state-of-the-art continual learning baselines, and it uses up to ten times fewer learnable parameters compared to baselines. We also make the case that sparsity is an essential ingredient for continual learning. The NISPA code is available at https://github.com/BurakGurbuz97/NISPA.
