论文标题
带有提示的模块化和参数效率的多模式融合
Modular and Parameter-Efficient Multimodal Fusion with Prompting
论文作者
论文摘要
最近的研究在大规模多模式预训练方面取得了令人印象深刻的进步。在模型大小的快速增长的背景下，有必要寻求除填充以外的有效和灵活的方法。在本文中，我们建议使用提示向量来对齐方式。我们的方法与低资源设置中其他几种多模式融合方法的性能相当。我们进一步表明，我们的方法是模块化和参数效率，用于处理涉及两种或多个数据模式的任务。
Recent research has made impressive progress in large-scale multimodal pre-training. In the context of the rapid growth of model size, it is necessary to seek efficient and flexible methods other than finetuning. In this paper, we propose to use prompt vectors to align the modalities. Our method achieves comparable performance to several other multimodal fusion methods in low-resource settings. We further show that our method is modular and parameter-efficient for processing tasks involving two or more data modalities.
