论文标题
注意差距：衡量多个目标的概括性能
Mind the Gap: Measuring Generalization Performance Across Multiple Objectives
论文作者
论文摘要
经常考虑到多个目标，例如最小化推理时间，同时也最大化准确性，通常会构建现代机器学习模型。多目标超参数优化（MHPO）算法返回此类候选模型，并使用Pareto Front的近似值来评估其性能。在实践中，我们还希望在从验证到测试集时测量概括。但是，某些模型可能不再是帕累托最佳的，这使得在测试集评估时如何量化MHPO方法的性能。为了解决这一问题，我们提供了一种新颖的评估协议，该方案允许测量MHPO方法的概括性能并研究其比较两个优化实验的能力。
Modern machine learning models are often constructed taking into account multiple objectives, e.g., minimizing inference time while also maximizing accuracy. Multi-objective hyperparameter optimization (MHPO) algorithms return such candidate models, and the approximation of the Pareto front is used to assess their performance. In practice, we also want to measure generalization when moving from the validation to the test set. However, some of the models might no longer be Pareto-optimal which makes it unclear how to quantify the performance of the MHPO method when evaluated on the test set. To resolve this, we provide a novel evaluation protocol that allows measuring the generalization performance of MHPO methods and studying its capabilities for comparing two optimization experiments.
