论文标题
神经特征映射的超平面边界
Hyperplane bounds for neural feature mappings
论文作者
论文摘要
深度学习方法使用诸如横熵损失之类的损失功能最小化经验风险。当最小化经验风险时，学习函数的概括仍然取决于训练数据的性能，vapnik-chervonenkis（VC）功能的限制和训练示例的数量。神经网络具有大量参数，这与它们的VC维度相关，通常是大但不是无限的，通常需要大量的培训实例来有效地训练它们。 在这项工作中，我们探讨了如何使用神经网络优化特征映射，以减少映射生成的空间中发现的超平面的有效VC维数。对这项研究结果的解释是，可以定义控制分离超平面的VC维度的损失。我们评估了这种方法，并观察到使用此方法时的性能会在训练集的大小很小时改善。
Deep learning methods minimise the empirical risk using loss functions such as the cross entropy loss. When minimising the empirical risk, the generalisation of the learnt function still depends on the performance on the training data, the Vapnik-Chervonenkis(VC)-dimension of the function and the number of training examples. Neural networks have a large number of parameters, which correlates with their VC-dimension that is typically large but not infinite, and typically a large number of training instances are needed to effectively train them. In this work, we explore how to optimize feature mappings using neural network with the intention to reduce the effective VC-dimension of the hyperplane found in the space generated by the mapping. An interpretation of the results of this study is that it is possible to define a loss that controls the VC-dimension of the separating hyperplane. We evaluate this approach and observe that the performance when using this method improves when the size of the training set is small.
