论文标题
仇恨言论和孟加拉语的令人反感的语言检测
Hate Speech and Offensive Language Detection in Bengali
论文作者
论文摘要
社交媒体通常是各种仇恨和令人反感的内容的繁殖地。在社交媒体上识别这种内容至关重要，因为它对不受约束的社会中的种族，性别或宗教的影响。但是，尽管在英语中进行了仇恨言论检测的广泛研究，但在孟加拉语等低资源语言中，令人讨厌的内容检测存在差距。此外，当前的社交媒体趋势是将罗马化孟加拉语用于定期互动。为了克服现有研究的局限性，在这项研究中，我们开发了一个带注释的数据集，该数据集由5K实际和5K罗马化的孟加拉语推文组成的10K孟加拉帖子。我们实施了几种基线模型来分类此类仇恨帖子。我们进一步探讨了室上转移机制以提高分类性能。最后，我们通过查看模型错误分类的帖子来执行深入的错误分析。在分别培训实际和罗马的数据集时，我们观察到XLM-Roberta表现最好。此外，我们目睹了在联合培训和几次训练中，Muril通过更好地解释语义表达来优于其他模型。我们将代码和数据集公开为他人。
Social media often serves as a breeding ground for various hateful and offensive content. Identifying such content on social media is crucial due to its impact on the race, gender, or religion in an unprejudiced society. However, while there is extensive research in hate speech detection in English, there is a gap in hateful content detection in low-resource languages like Bengali. Besides, a current trend on social media is the use of Romanized Bengali for regular interactions. To overcome the existing research's limitations, in this study, we develop an annotated dataset of 10K Bengali posts consisting of 5K actual and 5K Romanized Bengali tweets. We implement several baseline models for the classification of such hateful posts. We further explore the interlingual transfer mechanism to boost classification performance. Finally, we perform an in-depth error analysis by looking into the misclassified posts by the models. While training actual and Romanized datasets separately, we observe that XLM-Roberta performs the best. Further, we witness that on joint training and few-shot training, MuRIL outperforms other models by interpreting the semantic expressions better. We make our code and dataset public for others.
