论文标题
确定性解码用于变异自动编码器中的离散数据
Deterministic Decoding for Discrete Data in Variational Autoencoders
论文作者
论文摘要
变分自动编码器是用于建模离散数据的突出生成模型。但是，使用灵活的解码器，它们倾向于忽略潜在代码。在本文中，我们研究了具有确定性解码器（DD-VAE）的VAE模型的顺序数据，该数据选择了得分最高的令牌而不是采样。确定性解码仅依赖于潜在代码是产生各种物体的唯一方法，从而改善了学习的歧管的结构。为了实施DD-VAE，我们提出了一类新的有界支持提案分布，并为高斯和统一先验提供了Kullback-Leibler的差异。我们还研究确定性解码目标函数的持续放松，并分析重建精度和放松参数的关系。我们证明了在多个数据集上DD-VAE的性能，包括分子产生和优化问题。
Variational autoencoders are prominent generative models for modeling discrete data. However, with flexible decoders, they tend to ignore the latent codes. In this paper, we study a VAE model with a deterministic decoder (DD-VAE) for sequential data that selects the highest-scoring tokens instead of sampling. Deterministic decoding solely relies on latent codes as the only way to produce diverse objects, which improves the structure of the learned manifold. To implement DD-VAE, we propose a new class of bounded support proposal distributions and derive Kullback-Leibler divergence for Gaussian and uniform priors. We also study a continuous relaxation of deterministic decoding objective function and analyze the relation of reconstruction accuracy and relaxation parameters. We demonstrate the performance of DD-VAE on multiple datasets, including molecular generation and optimization problems.
