论文标题
当烟道遇到Flang时：金融领域的基准和大型预培训的语言模型
WHEN FLUE MEETS FLANG: Benchmarks and Large Pre-trained Language Model for Financial Domain
论文作者
论文摘要
预训练的语言模型在各种任务和域上表现出了令人印象深刻的表现。先前对金融语言模型的研究通常采用通用培训计划来培训标准模型架构，而无需完全利用财务数据的丰富性。我们提出了一种新颖的领域特定财务语言模型（FLANG），该模型使用财务关键字和短语，以更好地掩盖掩盖，以及跨度边界目标和填充目标。此外，该领域的评估基准受到限制。为此，我们贡献了金融语言理解评估（FLUE），这是金融领域的开源综合基准。其中包括金融领域5个NLP任务的新基准，以及先前研究中使用的常见基准。这些基准测试的实验表明，我们的模型表现优于先前文献中有关各种NLP任务的模型。我们的模型，代码和基准数据在GitHub和HuggingFace上公开可用。
Pre-trained language models have shown impressive performance on a variety of tasks and domains. Previous research on financial language models usually employs a generic training scheme to train standard model architectures, without completely leveraging the richness of the financial data. We propose a novel domain specific Financial LANGuage model (FLANG) which uses financial keywords and phrases for better masking, together with span boundary objective and in-filing objective. Additionally, the evaluation benchmarks in the field have been limited. To this end, we contribute the Financial Language Understanding Evaluation (FLUE), an open-source comprehensive suite of benchmarks for the financial domain. These include new benchmarks across 5 NLP tasks in financial domain as well as common benchmarks used in the previous research. Experiments on these benchmarks suggest that our model outperforms those in prior literature on a variety of NLP tasks. Our models, code and benchmark data are publicly available on Github and Huggingface.
