论文标题
从预先训练的扩散概率模型中学习无监督的表示
Unsupervised Representation Learning from Pre-trained Diffusion Probabilistic Models
论文作者
论文摘要
扩散概率模型（DPM）显示出生成高质量图像样本的强大能力。最近，已经提出了扩散自动编码器（diff-ae）来探索通过自动编码来表示的DPM。他们的关键想法是共同训练一个编码器，以发现图像中有意义的表示形式，并作为有条件的DPM作为重建图像的解码器。考虑到从头开始的DPM培训将需要很长时间，并且存在许多预先培训的DPM，我们建议\ textbf {p}重新培训\ textbf {d} pm \ textbf {a} uto \ textbf {uto \ textbf {e}图像重建，比DIFF-AE具有更好的训练效率和性能。具体而言，我们发现预训练的DPM无法从其潜在变量重建图像的原因是由于向前过程的信息丢失造成的，这导致其预测的后均值和真实图像之间的差距。从这个角度来看，可以将分类器指导的采样方法解释为计算额外的平均变化以填补空白，从而在样本中重建丢失的类信息。这些暗示差距对应于图像的丢失信息，我们可以通过填充间隙来重建图像。从中汲取灵感，我们采用可训练的模型来根据编码表示形式预测平均变化，并训练它以填补尽可能多的空白，以这种方式，编码器被迫从图像中学习尽可能多的信息以帮助填充。通过重复预先训练的DPM网络的一部分并重新设计扩散损失的加权方案，PDAE可以有效地从图像中学习有意义的表示。广泛的实验证明了PDAE的有效性，效率和灵活性。
Diffusion Probabilistic Models (DPMs) have shown a powerful capacity of generating high-quality image samples. Recently, diffusion autoencoders (Diff-AE) have been proposed to explore DPMs for representation learning via autoencoding. Their key idea is to jointly train an encoder for discovering meaningful representations from images and a conditional DPM as the decoder for reconstructing images. Considering that training DPMs from scratch will take a long time and there have existed numerous pre-trained DPMs, we propose \textbf{P}re-trained \textbf{D}PM \textbf{A}uto\textbf{E}ncoding (\textbf{PDAE}), a general method to adapt existing pre-trained DPMs to the decoders for image reconstruction, with better training efficiency and performance than Diff-AE. Specifically, we find that the reason that pre-trained DPMs fail to reconstruct an image from its latent variables is due to the information loss of forward process, which causes a gap between their predicted posterior mean and the true one. From this perspective, the classifier-guided sampling method can be explained as computing an extra mean shift to fill the gap, reconstructing the lost class information in samples. These imply that the gap corresponds to the lost information of the image, and we can reconstruct the image by filling the gap. Drawing inspiration from this, we employ a trainable model to predict a mean shift according to encoded representation and train it to fill as much gap as possible, in this way, the encoder is forced to learn as much information as possible from images to help the filling. By reusing a part of network of pre-trained DPMs and redesigning the weighting scheme of diffusion loss, PDAE can learn meaningful representations from images efficiently. Extensive experiments demonstrate the effectiveness, efficiency and flexibility of PDAE.
