论文标题
学习有限的玻尔兹曼机器，具有稀疏的潜在变量
Learning Restricted Boltzmann Machines with Sparse Latent Variables
论文作者
论文摘要
限制性玻尔兹曼机器（RBMS）是具有潜在变量的无向图形模型的常见家族。 RBM通过两分图描述，所有观察到的变量在一个层中，另一层中的所有潜在变量。我们考虑根据根据它生成的样品学习RBM的任务。该任务的最佳算法目前具有时间复杂性$ \ tilde {o}（n^2）$用于Ferromagnetic RBMS（即具有有吸引力的潜力），但$ \ tilde {o}（o}（n^d）$对于一般rbms，一般rbms，如果$ n $是$ n $是$ n $是$ n $的最大变量。让观察到的变量的MRF邻域是观察到的变量边缘分布的马尔可夫随机场中的邻域。在本文中，我们给出了一种用于学习通用RBM的算法，并使用时间复杂性$ \ tilde {o}（n^{2^s+1}）$，其中$ s $是连接到观察到变量的MRF社区的最大潜在变量数量。当$ s <\ log_2（d-1）$对应于具有稀疏潜在变量的RBM时，这是一个改进。此外，我们给出了该学习算法的一个版本，该算法恢复了一个较小的预测误差的模型，其样品复杂性与观察到的变量的马尔可夫随机场中的最小潜力无关。这很有趣，因为当前算法的样品复杂性与最小电位的倒数相反，这是不受RBM的自然性质来控制的。
Restricted Boltzmann Machines (RBMs) are a common family of undirected graphical models with latent variables. An RBM is described by a bipartite graph, with all observed variables in one layer and all latent variables in the other. We consider the task of learning an RBM given samples generated according to it. The best algorithms for this task currently have time complexity $\tilde{O}(n^2)$ for ferromagnetic RBMs (i.e., with attractive potentials) but $\tilde{O}(n^d)$ for general RBMs, where $n$ is the number of observed variables and $d$ is the maximum degree of a latent variable. Let the MRF neighborhood of an observed variable be its neighborhood in the Markov Random Field of the marginal distribution of the observed variables. In this paper, we give an algorithm for learning general RBMs with time complexity $\tilde{O}(n^{2^s+1})$, where $s$ is the maximum number of latent variables connected to the MRF neighborhood of an observed variable. This is an improvement when $s < \log_2 (d-1)$, which corresponds to RBMs with sparse latent variables. Furthermore, we give a version of this learning algorithm that recovers a model with small prediction error and whose sample complexity is independent of the minimum potential in the Markov Random Field of the observed variables. This is of interest because the sample complexity of current algorithms scales with the inverse of the minimum potential, which cannot be controlled in terms of natural properties of the RBM.
