论文标题
关于大约贝叶斯推断的批发归一化
On Batch Normalisation for Approximate Bayesian Inference
论文作者
论文摘要
我们研究贝叶斯神经网络中的变异推理方法（例如平均场或MC脱落）的批归归术。我们表明，批处理范围不会影响证据下限（ELBO）的最佳。此外，我们研究了蒙特卡洛批准归一化（MCBN）算法，该算法被认为是与MC辍学平行的近似推理技术，并表明，对于较大的批次大小，MCBN无法捕获认识论的不确定性。最后，我们提供有关解决此故障所需的内容的见解，即必须将迷你批量大小视为MCBN中的变异参数。我们对ELBO的渐近参数发表评论，这表明随着数据集的大小朝着无穷大的增加，批处理大小也必须向Infinity增加，而MCBN也必须成为有效的近似推理技术。
We study batch normalisation in the context of variational inference methods in Bayesian neural networks, such as mean-field or MC Dropout. We show that batch-normalisation does not affect the optimum of the evidence lower bound (ELBO). Furthermore, we study the Monte Carlo Batch Normalisation (MCBN) algorithm, proposed as an approximate inference technique parallel to MC Dropout, and show that for larger batch sizes, MCBN fails to capture epistemic uncertainty. Finally, we provide insights into what is required to fix this failure, namely having to view the mini-batch size as a variational parameter in MCBN. We comment on the asymptotics of the ELBO with respect to this variational parameter, showing that as dataset size increases towards infinity, the batch-size must increase towards infinity as well for MCBN to be a valid approximate inference technique.
