论文标题
锁：用户对私有和联合的最佳客户端采样
LOCKS: User Differentially Private and Federated Optimal Client Sampling
论文作者
论文摘要
随着隐私法律的变化，通常很难要求客户数据保留在设备上，而不是发送到服务器。因此，大多数处理都会在设备上发生，并且仅将更改的元素发送到服务器。这种机制是通过利用差异隐私和联合学习来开发的。差异隐私为客户端输出增加了噪音，从而恶化了每次迭代的质量。这种分布式设置增加了一层复杂性，以及其他交流和性能开销。这些成本是每回合的加性，因此我们需要减少迭代次数。在这项工作中，我们提供了一个分析框架，用于研究基于梯度的分布式算法的收敛保证。我们表明，我们的私人算法将预期的梯度差异最小化了大约$ d^2 $ rounds，其中d是模型的维度。我们讨论并提出了提高收敛速率的新方法，以使用重要性采样（IS）和梯度多样性来最大程度地减少开销。最后，我们提供的替代框架可能更适合利用诸如IS和梯度多样性之类的客户抽样技术。
With changes in privacy laws, there is often a hard requirement for client data to remain on the device rather than being sent to the server. Therefore, most processing happens on the device, and only an altered element is sent to the server. Such mechanisms are developed by leveraging differential privacy and federated learning. Differential privacy adds noise to the client outputs and thus deteriorates the quality of each iteration. This distributed setting adds a layer of complexity and additional communication and performance overhead. These costs are additive per round, so we need to reduce the number of iterations. In this work, we provide an analytical framework for studying the convergence guarantees of gradient-based distributed algorithms. We show that our private algorithm minimizes the expected gradient variance by approximately $d^2$ rounds, where d is the dimensionality of the model. We discuss and suggest novel ways to improve the convergence rate to minimize the overhead using Importance Sampling (IS) and gradient diversity. Finally, we provide alternative frameworks that might be better suited to exploit client sampling techniques like IS and gradient diversity.
