论文标题
机器学习的逻辑
Logic of Machine Learning
论文作者
论文摘要
主要问题是：为什么以及如何根据有限样本进行预测？统计学习理论没有回答这个问题。在这里，我建议预测需要对基本依赖的“可预测性”的信念，并且学习涉及搜索一个假设，在这种假设下，根据观察结果，这些信念最少。对于给定数据，假设和特定类型的可预测性信念的这些违规（“错误”）的度量被形式化为观察和假设的模态逻辑（LOH）中不一致的概念。我展示了许多流行的教科书学习者的示例（从层次聚类到K-NN和SVM），每个人都可以最大程度地减少其自己的不一致版本。此外，不一致性的概念足以使某些重要的数据分析问题形式化，而不是被视为ML的一部分。
The main question is: why and how can we ever predict based on a finite sample? The question is not answered by statistical learning theory. Here, I suggest that prediction requires belief in "predictability" of the underlying dependence, and learning involves search for a hypothesis where these beliefs are violated the least given the observations. The measure of these violations ("errors") for given data, hypothesis and particular type of predictability beliefs is formalized as concept of incongruity in modal Logic of Observations and Hypotheses (LOH). I show on examples of many popular textbook learners (from hierarchical clustering to k-NN and SVM) that each of them minimizes its own version of incongruity. In addition, the concept of incongruity is shown to be flexible enough for formalization of some important data analysis problems, not considered as part of ML.
