论文标题
通过$ m $ th信息密度的中心时刻的概括错误界定
Generalization Error Bounds via $m$th Central Moments of the Information Density
论文作者
论文摘要
我们提出了一种对随机学习算法的概括误差的界限的通用方法。我们的方法可用于在平均概括误差以及其尾巴概率上的界限上获得界限，这两种情况都可以随机生成新假设时，每次使用算法时都会随机生成 - 在可能的（PAC） - 贝斯（PAC） - 贝斯（PAC） - 贝斯文学中通常都假定的是单抽出的情况 - 在单一绘制的情况下，仅提取了一次假设。在最后的情况下，我们提出了一种新颖的界限，该界限是在信息密度的中心时刻明确的。结合表明，可以控制的信息密度力矩的顺序越高，限制对所需置信度的概括的依赖性较小。此外，我们使用二进制假设测试的工具来得出第二个界限，该界限在信息密度的尾部是显式的。这种结合证实，信息密度的尾巴的快速衰减产生了对置信度限制的概括的依赖性。
We present a general approach to deriving bounds on the generalization error of randomized learning algorithms. Our approach can be used to obtain bounds on the average generalization error as well as bounds on its tail probabilities, both for the case in which a new hypothesis is randomly generated every time the algorithm is used - as often assumed in the probably approximately correct (PAC)-Bayesian literature - and in the single-draw case, where the hypothesis is extracted only once. For this last scenario, we present a novel bound that is explicit in the central moments of the information density. The bound reveals that the higher the order of the information density moment that can be controlled, the milder the dependence of the generalization bound on the desired confidence level. Furthermore, we use tools from binary hypothesis testing to derive a second bound, which is explicit in the tail of the information density. This bound confirms that a fast decay of the tail of the information density yields a more favorable dependence of the generalization bound on the confidence level.
