论文标题
奥卡姆的幽灵
Occam's Ghost
论文作者
论文摘要
本文通过查找具有最小数量位的模型，将OCCAM剃须刀的原理应用于非参数模型构建统计数据，从而导致了概率密度估计器的异常有效的正则化方法。这个想法源于以下事实：可能性最大化还可以最大程度地减少编码数据集所需的位数。但是，传统方法忽略了模型参数的优化也可能无意中发挥了编码数据点的作用。本文显示了如何将位计数扩展到模型参数，从而为参数模型提供了第一个真正的复杂性度量。最小化数据集模型的总钻头需求有利于较小的导数，更平滑的概率密度函数估计，最重要的是，相关参数较少的相位空间。实际上，它能够同时使用修剪参数并以较小的概率检测特征。还显示了如何将其应用于任何平滑的非参数概率密度估计器。
This article applies the principle of Occam's Razor to non-parametric model building of statistical data, by finding a model with the minimal number of bits, leading to an exceptionally effective regularization method for probability density estimators. The idea comes from the fact that likelihood maximization also minimizes the number of bits required to encode a dataset. However, traditional methods overlook that the optimization of model parameters may also inadvertently play the part in encoding data points. The article shows how to extend the bit counting to the model parameters as well, providing the first true measure of complexity for parametric models. Minimizing the total bit requirement of a model of a dataset favors smaller derivatives, smoother probability density function estimates and most importantly, a phase space with fewer relevant parameters. In fact, it is able prune parameters and detect features with small probability at the same time. It is also shown, how it can be applied to any smooth, non-parametric probability density estimator.
