论文标题
生成对抗网络的收敛动力学：双度度流量
Convergence dynamics of Generative Adversarial Networks: the dual metric flows
论文作者
论文摘要
拟合神经网络通常求助于随机（或类似）梯度下降，这是梯度下降动力学的耐噪声（且有效）的分辨率。它输出一系列网络参数，这些参数在训练步骤中会演变。梯度下降是极限，当学习率较小并且批处理大小是无限的，这是在训练期间获得的这组越来越最佳的网络参数。在此贡献中，我们改为研究机器学习中使用的生成对抗网络中的收敛性。我们研究了少量学习率的限制，并表明，与单个网络培训类似，GAN学习动力趋于消失，从而消失了一定的限制动态。这导致我们考虑度量空间中的演化方程（这是我们称为双流量的自然框架）。我们给出了解决方案的正式定义，并证明了融合。然后将该理论应用于gan的特定实例，我们讨论了这种洞察力如何帮助理解和减轻模式崩溃。 关键字：gan;度量流；生成网络
Fitting neural networks often resorts to stochastic (or similar) gradient descent which is a noise-tolerant (and efficient) resolution of a gradient descent dynamics. It outputs a sequence of networks parameters, which sequence evolves during the training steps. The gradient descent is the limit, when the learning rate is small and the batch size is infinite, of this set of increasingly optimal network parameters obtained during training. In this contribution, we investigate instead the convergence in the Generative Adversarial Networks used in machine learning. We study the limit of small learning rate, and show that, similar to single network training, the GAN learning dynamics tend, for vanishing learning rate to some limit dynamics. This leads us to consider evolution equations in metric spaces (which is the natural framework for evolving probability laws) that we call dual flows. We give formal definitions of solutions and prove the convergence. The theory is then applied to specific instances of GANs and we discuss how this insight helps understand and mitigate the mode collapse. Keywords: GAN; metric flow; generative network
