论文标题
具有连续数据更新的模型稳定性
Model Stability with Continuous Data Updates
论文作者
论文摘要
在本文中，我们在具有连续培训数据更新的较大，复杂的NLP系统的背景下研究机器学习（ML）模型的“稳定性”。在这项研究中，我们提出了一种评估模型稳定性的方法（在各种实验条件下我们称为抖动。我们发现，模型设计选择，包括网络体系结构和输入表示，通过对四个文本分类任务和两个序列标签的实验对稳定性产生了关键的影响。在分类任务中，基于非RNN的模型在分类任务中更稳定，而不是更稳定的模型，而不是更稳定的稳定性，而不是稳定的稳定性。稳定的标签任务。
In this paper, we study the "stability" of machine learning (ML) models within the context of larger, complex NLP systems with continuous training data updates. For this study, we propose a methodology for the assessment of model stability (which we refer to as jitter under various experimental conditions. We find that model design choices, including network architecture and input representation, have a critical impact on stability through experiments on four text classification tasks and two sequence labeling tasks. In classification tasks, non-RNN-based models are observed to be more stable than RNN-based ones, while the encoder-decoder model is less stable in sequence labeling tasks. Moreover, input representations based on pre-trained fastText embeddings contribute to more stability than other choices. We also show that two learning strategies -- ensemble models and incremental training -- have a significant influence on stability. We recommend ML model designers account for trade-offs in accuracy and jitter when making modeling choices.
