论文标题
概率序列建模的张量网络
Tensor Networks for Probabilistic Sequence Modeling
论文作者
论文摘要
张量网络是为计算多体物理学开发的强大建模框架，直到最近才应用于机器学习中。在这项工作中，我们利用统一的矩阵乘积状态（U-MP）模型来序列数据的概率建模。我们首先表明U-MP启用了序列级并行性，可以在深度O中评估长度-N序列（log n）。然后，我们引入了一种新颖的生成算法，使训练有素的U-MP有效地从各种条件分布中进行采样，每个分布都由正则表达式定义。该算法的特殊情况对应于自回旋和填充的抽样，但是更复杂的正则表达式可以以神经生成模型中没有直接模拟的方式生成丰富的结构化数据。使用合成和真实文本数据进行序列建模的实验表明，U-MP的表现优于各种基准，并在存在有限数据的情况下有效地概括了其预测。
Tensor networks are a powerful modeling framework developed for computational many-body physics, which have only recently been applied within machine learning. In this work we utilize a uniform matrix product state (u-MPS) model for probabilistic modeling of sequence data. We first show that u-MPS enable sequence-level parallelism, with length-n sequences able to be evaluated in depth O(log n). We then introduce a novel generative algorithm giving trained u-MPS the ability to efficiently sample from a wide variety of conditional distributions, each one defined by a regular expression. Special cases of this algorithm correspond to autoregressive and fill-in-the-blank sampling, but more complex regular expressions permit the generation of richly structured data in a manner that has no direct analogue in neural generative models. Experiments on sequence modeling with synthetic and real text data show u-MPS outperforming a variety of baselines and effectively generalizing their predictions in the presence of limited data.
