论文标题
没有提示的有效的几次学习
Efficient Few-Shot Learning Without Prompts
论文作者
论文摘要
最近的几种方法，例如参数有效的微调（PEFT）和模式开发培训（PET），在标签降低设置中取得了令人印象深刻的结果。但是，它们很难使用，因为它们会因手动制作的提示带来很大的可变性，并且通常需要十亿参数语言模型才能达到高精度。为了解决这些缺点，我们提出了setFit（句子变形金刚微调），这是一个有效且及时的框架，用于对句子变形金刚（ST）进行微调微调。 SetFit首先以对比的暹罗方式对少数文本对进行审计的ST进行微调。然后，将所得模型用于生成丰富的文本嵌入，该嵌入方式用于训练分类头。这个简单的框架不需要任何提示或口头化，并且比现有技术少的参数较少，因此可以实现高精度。我们的实验表明，SetFit通过PEFT和PET技术获得了可比的结果，而训练的速度更快。我们还表明，SETFIT可以通过简单地切换ST主体来应用于多语言设置。我们的代码可从https://github.com/huggingface/setFit以及我们的数据集获得，网址为https://huggingface.co/setfit。
Recent few-shot methods, such as parameter-efficient fine-tuning (PEFT) and pattern exploiting training (PET), have achieved impressive results in label-scarce settings. However, they are difficult to employ since they are subject to high variability from manually crafted prompts, and typically require billion-parameter language models to achieve high accuracy. To address these shortcomings, we propose SetFit (Sentence Transformer Fine-tuning), an efficient and prompt-free framework for few-shot fine-tuning of Sentence Transformers (ST). SetFit works by first fine-tuning a pretrained ST on a small number of text pairs, in a contrastive Siamese manner. The resulting model is then used to generate rich text embeddings, which are used to train a classification head. This simple framework requires no prompts or verbalizers, and achieves high accuracy with orders of magnitude less parameters than existing techniques. Our experiments show that SetFit obtains comparable results with PEFT and PET techniques, while being an order of magnitude faster to train. We also show that SetFit can be applied in multilingual settings by simply switching the ST body. Our code is available at https://github.com/huggingface/setfit and our datasets at https://huggingface.co/setfit .
