论文标题
出乎意料的有用：收敛范围和现实世界分布式学习
Unexpectedly Useful: Convergence Bounds And Real-World Distributed Learning
论文作者
论文摘要
在运行任务本身之前，收敛范围是获取分布式机器学习任务性能的信息的主要工具之一。在这项工作中，我们执行了一组实验，以评估在哪种程度上，以何种方式可以预测和改善现实世界分布式（即联合）学习任务的性能。我们发现，正如鉴于其获得的方式可以预期的那样，边界非常松散，它们的相对幅度反映了训练而不是测试损失。更出乎意料的是，我们发现在界限中出现的一些数量对于确定最有可能有助于学习过程的客户非常有用，而无需披露有关其数据集质量或大小的任何信息。这表明，有必要对可以利用融合界限来提高现实世界分布式学习任务的性能的方式进行进一步的研究。
Convergence bounds are one of the main tools to obtain information on the performance of a distributed machine learning task, before running the task itself. In this work, we perform a set of experiments to assess to which extent, and in which way, such bounds can predict and improve the performance of real-world distributed (namely, federated) learning tasks. We find that, as can be expected given the way they are obtained, bounds are quite loose and their relative magnitude reflects the training rather than the testing loss. More unexpectedly, we find that some of the quantities appearing in the bounds turn out to be very useful to identify the clients that are most likely to contribute to the learning process, without requiring the disclosure of any information about the quality or size of their datasets. This suggests that further research is warranted on the ways -- often counter-intuitive -- in which convergence bounds can be exploited to improve the performance of real-world distributed learning tasks.
