论文标题
关于gan中梯度下降的收敛：MMD gan作为梯度流
On the Convergence of Gradient Descent in GANs: MMD GAN As a Gradient Flow
论文作者
论文摘要
我们考虑最大平均差异（$ \ mathrm {mmd} $）gan问题，并提出了一个参数内核化的梯度流，该流程模仿了梯度正规化$ \ mathrm {mmd} $ gan中的Min-Max游戏。我们表明，该流提供了一个下降方向，最大程度地将概率分布的统计歧管上的$ \ mathrm {mmd} $最小化。然后，我们得出一个明确的条件，该条件可确保在梯度正规化的$ \ mathrm {mmd} $ gan中的发电机参数空间上的梯度下降是全球收敛到目标分布的。在这种情况下，我们给出MMD GAN梯度下降的非渐近收敛结果。本文的另一个贡献是引入了$ \ mathrm {mmd} $的正则化的动态公式，并证明了$ \ mathrm {mmd} $的参数核下降是此功能相对于新的Riemannian结构的梯度流。我们获得的理论结果使人们可以治疗相当一般的功能的梯度流，因此在GAN以外的统计歧管上对其他类型的变异推断具有潜在的应用。最后，数值实验表明，我们的参数核梯度流稳定了GAN训练并确保收敛。
We consider the maximum mean discrepancy ($\mathrm{MMD}$) GAN problem and propose a parametric kernelized gradient flow that mimics the min-max game in gradient regularized $\mathrm{MMD}$ GAN. We show that this flow provides a descent direction minimizing the $\mathrm{MMD}$ on a statistical manifold of probability distributions. We then derive an explicit condition which ensures that gradient descent on the parameter space of the generator in gradient regularized $\mathrm{MMD}$ GAN is globally convergent to the target distribution. Under this condition, we give non asymptotic convergence results of gradient descent in MMD GAN. Another contribution of this paper is the introduction of a dynamic formulation of a regularization of $\mathrm{MMD}$ and demonstrating that the parametric kernelized descent for $\mathrm{MMD}$ is the gradient flow of this functional with respect to the new Riemannian structure. Our obtained theoretical result allows ones to treat gradient flows for quite general functionals and thus has potential applications to other types of variational inferences on a statistical manifold beyond GANs. Finally, numerical experiments suggest that our parametric kernelized gradient flow stabilizes GAN training and guarantees convergence.
