论文标题
SDMUSE：随机差异音乐编辑和通过混合表示
SDMuse: Stochastic Differential Music Editing and Generation via Hybrid Representation
论文作者
论文摘要
虽然深层生成模型已经赋予了音乐的能力，但以精细的粒度编辑现有的音乐作品仍然是一个具有挑战性且探索不足的问题。在本文中，我们提出了SDMUSE，这是一种统一的随机差异音乐编辑和生成框架，它不仅可以从头开始构成整个音乐作品，而且还可以通过许多方式修改现有音乐作品，例如组合，延续，插入，介入和风格转移。拟议的SDMUSE遵循两阶段的管道，以在包括钢琴和Midi-event在内的混合代表制度上实现音乐发电和编辑。特别是，SDMuse首先通过基于扩散模型生成的随机微分方程（SDE）进行迭代性来生成/编辑钢琴，然后对生成的钢琴进行改进，并预测中等事件的图表自动注册。我们在AILABS1K7流行音乐数据集上评估了我们方法的生成音乐，以各种音乐编辑和发电任务的质量和可控性。实验结果证明了我们提出的随机差异音乐编辑和发电过程以及混合表示的有效性。
While deep generative models have empowered music generation, it remains a challenging and under-explored problem to edit an existing musical piece at fine granularity. In this paper, we propose SDMuse, a unified Stochastic Differential Music editing and generation framework, which can not only compose a whole musical piece from scratch, but also modify existing musical pieces in many ways, such as combination, continuation, inpainting, and style transferring. The proposed SDMuse follows a two-stage pipeline to achieve music generation and editing on top of a hybrid representation including pianoroll and MIDI-event. In particular, SDMuse first generates/edits pianoroll by iteratively denoising through a stochastic differential equation (SDE) based on a diffusion model generative prior, and then refines the generated pianoroll and predicts MIDI-event tokens auto-regressively. We evaluate the generated music of our method on ailabs1k7 pop music dataset in terms of quality and controllability on various music editing and generation tasks. Experimental results demonstrate the effectiveness of our proposed stochastic differential music editing and generation process, as well as the hybrid representations.
