论文标题
部分可观测时空混沌系统的无模型预测
FLECS-CGD: A Federated Learning Second-Order Framework via Compression and Sketching with Compressed Gradient Differences
论文作者
论文摘要
在最近的论文Flecs（Agafonov等人，Flecs：通过压缩和素描的联邦学习二阶框架）中，为联邦学习问题提出了二阶框架Flecs。这种方法利用素描的黑森人的压缩来使沟通成本较低。然而，烟叶的主要瓶颈是梯度通信而没有压缩。在本文中，我们提出了具有压缩梯度差异的Flecs的修饰，我们称之为FLECS-CGD（具有压缩梯度差异的Flecs），并使其适用于随机优化。在强烈凸和非凸案件中提供了收敛保证。实验显示了拟议方法的实际好处。
In the recent paper FLECS (Agafonov et al, FLECS: A Federated Learning Second-Order Framework via Compression and Sketching), the second-order framework FLECS was proposed for the Federated Learning problem. This method utilize compression of sketched Hessians to make communication costs low. However, the main bottleneck of FLECS is gradient communication without compression. In this paper, we propose the modification of FLECS with compressed gradient differences, which we call FLECS-CGD (FLECS with Compressed Gradient Differences) and make it applicable for stochastic optimization. Convergence guarantees are provided in strongly convex and nonconvex cases. Experiments show the practical benefit of proposed approach.
