论文标题
从低级别近似的神经费舍尔内核中学习表示
Learning Representation from Neural Fisher Kernel with Low-rank Approximation
论文作者
论文摘要
在本文中，我们从内核的角度研究了神经网络的表示。我们首先定义了神经渔民内核（NFK），即用于神经网络的Fisher内核。我们表明，可以针对受监督和无监督的学习模型计算NFK，该模型可以用作表示提取的统一工具。此外，我们表明实用的NFK表现出低级结构。然后，我们提出了一种有效的算法，该算法计算NFK的低等级近似值，该算法将其扩展到大型数据集和网络。我们表明，从无监督的生成模型和监督的学习模型得出的NFK的低级别近似可提高数据的高质量紧凑表示，从而在各种机器学习任务上取得了竞争成果。
In this paper, we study the representation of neural networks from the view of kernels. We first define the Neural Fisher Kernel (NFK), which is the Fisher Kernel applied to neural networks. We show that NFK can be computed for both supervised and unsupervised learning models, which can serve as a unified tool for representation extraction. Furthermore, we show that practical NFKs exhibit low-rank structures. We then propose an efficient algorithm that computes a low rank approximation of NFK, which scales to large datasets and networks. We show that the low-rank approximation of NFKs derived from unsupervised generative models and supervised learning models gives rise to high-quality compact representations of data, achieving competitive results on a variety of machine learning tasks.
