论文标题
结构化及时调整
Structured Prompt Tuning
论文作者
论文摘要
我们提出了结构化及时调整，这是一种简单有效的方法，可以改善及时调整。我们没有将一系列可调嵌入到输入中，而是通过超网络生成软提示嵌入。我们的方法包含标准及时调整，可以在模型设计方面具有更大的灵活性，并且可以应用于单任务和多任务训练设置。从经验上讲，结构化及时调整在胶水基准上显示出 +1.2 $ 〜1.5点的增益，与标准及时调整相比，对学习率的变化不太敏感。
We propose structured prompt tuning, a simple and effective method to improve prompt tuning. Instead of prepending a sequence of tunable embeddings to the input, we generate the soft prompt embeddings through a hypernetwork. Our approach subsumes the standard prompt tuning, allows more flexibility in model design and can be applied to both single-task and multi-task training settings. Empirically, structured prompt tuning shows a gain of +1.2$~1.5 points on the GLUE benchmark and is less sensitive to the change of learning rate, compared to standard prompt tuning.
