论文标题
插值压缩参数子空间
Interpolating Compressed Parameter Subspaces
论文作者
论文摘要
受到神经子空间和模式连接性的最新工作的启发，我们重新访问了转移和/或插值输入分布的参数子空间采样（而不是单个未降低的分布）。我们在映射到一组火车时间分布的一组训练的参数上强制执行压缩的几何结构，将所得子空间表示为压缩参数子空间（CPS）。我们显示了其最佳参数位于CPS的转移分布类型的成功和故障模式。我们发现，CPS内的分类点估计物可以在一系列测试时间分布中产生高平均精度，包括后门，对抗，置换，风格化和旋转扰动。我们还发现，CP可以包含用于各种任务偏移的低损坏点估计（尽管插值，扰动，看不见或非相同的粗制标签）。我们进一步在使用CIFAR100的持续学习环境中证明了这一属性。
Inspired by recent work on neural subspaces and mode connectivity, we revisit parameter subspace sampling for shifted and/or interpolatable input distributions (instead of a single, unshifted distribution). We enforce a compressed geometric structure upon a set of trained parameters mapped to a set of train-time distributions, denoting the resulting subspaces as Compressed Parameter Subspaces (CPS). We show the success and failure modes of the types of shifted distributions whose optimal parameters reside in the CPS. We find that ensembling point-estimates within a CPS can yield a high average accuracy across a range of test-time distributions, including backdoor, adversarial, permutation, stylization and rotation perturbations. We also find that the CPS can contain low-loss point-estimates for various task shifts (albeit interpolated, perturbed, unseen or non-identical coarse labels). We further demonstrate this property in a continual learning setting with CIFAR100.
