论文标题
高斯 - 伯努利rbms没有眼泪
Gaussian-Bernoulli RBMs Without Tears
论文作者
论文摘要
我们重新审视了培训高斯 - 伯努利（Gaussian-Bernoulli）限制的鲍尔茨曼机器（GRBMS）的挑战性问题，并引入了两项创新。我们提出了一种新型的Gibbs-Langevin采样算法，该算法的表现优于诸如Gibbs采样的现有方法。我们提出了一种修改的对比差异（CD）算法，以便可以从噪声开始以GRBMS生成图像。这可以直接比较GRBM与深层生成模型，从而改善了RBM文献中的评估协议。此外，我们表明经过修改的CD和梯度剪辑足以以较高的学习率训练GRBM，从而消除了文献中各种技巧的必要性。尽管具有单隐藏式建筑，但关于高斯混合物，MNIST，FashionMnist和Celeba Show Grbms的实验仍可以生成好样品。我们的代码在：\ url {https://github.com/lrjconan/grbm}上发布。
We revisit the challenging problem of training Gaussian-Bernoulli restricted Boltzmann machines (GRBMs), introducing two innovations. We propose a novel Gibbs-Langevin sampling algorithm that outperforms existing methods like Gibbs sampling. We propose a modified contrastive divergence (CD) algorithm so that one can generate images with GRBMs starting from noise. This enables direct comparison of GRBMs with deep generative models, improving evaluation protocols in the RBM literature. Moreover, we show that modified CD and gradient clipping are enough to robustly train GRBMs with large learning rates, thus removing the necessity of various tricks in the literature. Experiments on Gaussian Mixtures, MNIST, FashionMNIST, and CelebA show GRBMs can generate good samples, despite their single-hidden-layer architecture. Our code is released at: \url{https://github.com/lrjconan/GRBM}.
