论文标题
微小分解器：用于语音分离的小时域变压器网络
Tiny-Sepformer: A Tiny Time-Domain Transformer Network for Speech Separation
论文作者
论文摘要
时间域变压器神经网络已证明了它们在语音分离任务中的优势。但是，这些模型通常具有大量的网络参数，因此经常遇到GPU内存爆炸问题。在本文中，我们提出了Tiny-Sepformer，这是用于语音分离的Transformer网络的微小版本。我们提出了两种减少模型参数和内存消耗的技术：（1）卷积 - 注意（CA）块，将香草变压器拼写为两个路径，多头注意力和1D深度可分开的卷积，（2）参数共享，在CA块中共享层参数。在我们的实验中，微小的隔离器可以大大降低模型大小，并在WSJ0-2/3MIX数据集上与香草sepformer实现可比的分离性能。
Time-domain Transformer neural networks have proven their superiority in speech separation tasks. However, these models usually have a large number of network parameters, thus often encountering the problem of GPU memory explosion. In this paper, we proposed Tiny-Sepformer, a tiny version of Transformer network for speech separation. We present two techniques to reduce the model parameters and memory consumption: (1) Convolution-Attention (CA) block, spliting the vanilla Transformer to two paths, multi-head attention and 1D depthwise separable convolution, (2) parameter sharing, sharing the layer parameters within the CA block. In our experiments, Tiny-Sepformer could greatly reduce the model size, and achieves comparable separation performance with vanilla Sepformer on WSJ0-2/3Mix datasets.
