论文标题
使用GECO和L0正则化的VAE瓶颈动态缩小
Dynamic Narrowing of VAE Bottlenecks Using GECO and L0 Regularization
论文作者
论文摘要
在设计变异自动编码器（VAE）或其他类型的潜在空间模型时，通常会预先定义潜在空间的维度。在此过程中，对于手头的应用程序，尺寸的数量可能不足或过度应用。如果维度不是预定义的，则通常使用耗时和资源的交叉验证来确定此参数。由于这些原因，我们已经开发了一种技术，可以使用广义ELBO自动且在训练期间自动缩小具有约束优化（GECO）和$ L_0 $ -AGEMMENT-AUGMENT-AREFORCE-MERGE（$ L_0 $ -ARM-ARM-ARM-ARM-ARM）梯度估计器的潜在空间维度。 GECO优化器可确保我们不会在重建误差上违反预定义的上限。本文介绍了我们方法的算法细节以及五个不同数据集的实验结果。我们发现我们的培训程序是稳定的，并且可以在不违反GECO约束的情况下有效地修剪潜在空间。
When designing variational autoencoders (VAEs) or other types of latent space models, the dimensionality of the latent space is typically defined upfront. In this process, it is possible that the number of dimensions is under- or overprovisioned for the application at hand. In case the dimensionality is not predefined, this parameter is usually determined using time- and resource-consuming cross-validation. For these reasons we have developed a technique to shrink the latent space dimensionality of VAEs automatically and on-the-fly during training using Generalized ELBO with Constrained Optimization (GECO) and the $L_0$-Augment-REINFORCE-Merge ($L_0$-ARM) gradient estimator. The GECO optimizer ensures that we are not violating a predefined upper bound on the reconstruction error. This paper presents the algorithmic details of our method along with experimental results on five different datasets. We find that our training procedure is stable and that the latent space can be pruned effectively without violating the GECO constraints.
