论文标题
适应不变性的分解和套索型对比度学习
Invariance-adapted decomposition and Lasso-type contrastive learning
论文作者
论文摘要
近年来，对比度学习在获取对解释和下游任务有用的数据集的表示方面的有效性。但是，描述这种有效性的机制尚未得到彻底分析，并且已经进行了许多研究来研究对比度学习捕获的数据结构。特别是，最近对\ citet {content_isaly}的研究表明，对比度学习能够将数据空间分解到所有增强及其补充的空间中。在本文中，我们介绍了适应不变性的潜在空间的概念，该空间将数据空间分解为每个增强及其补充的不变空间的交叉点。该分解概括了\ citet {content_isaly}中引入的分解，并描述了一种类似于组的谐波分析中的频率的结构。我们通过实验表明，使用套索型度量的对比度学习可用于寻找适应不变性的潜在空间，从而提出对比度学习的新潜力。我们还研究了何时可以将这种潜在空间识别为每个组件中的混合。
Recent years have witnessed the effectiveness of contrastive learning in obtaining the representation of dataset that is useful in interpretation and downstream tasks. However, the mechanism that describes this effectiveness have not been thoroughly analyzed, and many studies have been conducted to investigate the data structures captured by contrastive learning. In particular, the recent study of \citet{content_isolate} has shown that contrastive learning is capable of decomposing the data space into the space that is invariant to all augmentations and its complement. In this paper, we introduce the notion of invariance-adapted latent space that decomposes the data space into the intersections of the invariant spaces of each augmentation and their complements. This decomposition generalizes the one introduced in \citet{content_isolate}, and describes a structure that is analogous to the frequencies in the harmonic analysis of a group. We experimentally show that contrastive learning with lasso-type metric can be used to find an invariance-adapted latent space, thereby suggesting a new potential for the contrastive learning. We also investigate when such a latent space can be identified up to mixings within each component.
