论文标题
带有空中计算的分散SGD
Decentralized SGD with Over-the-Air Computation
论文作者
论文摘要
我们研究了无线网络中分散的随机梯度下降（DSGD）的性能，在该网络中，节点使用其本地数据集协作优化了目标函数。与传统的环境不同，节点通过无错误的正交通信链接进行通信，我们假设传输容易受到加性噪声和干扰。我们首先考虑一种点对点（P2P）的传输策略，称为OAC-P2P方案，该方案将节点对安排在正交子中以最小化的互换。由于在DSGD框架中，每个节点都需要在共识步骤中相邻模型的线性组合，然后我们提出了OAC-MAC方案，该方案利用无线介质的信号叠加属性来实现空气计算（OAC）。对于这两个方案，我们都将调度问题作为图形着色问题。我们从数值上评估了这两个方案在各种网络条件下的MNIST图像分类任务的性能。我们表明，OAC-MAC方案通过更少的沟通回合获得了更好的收敛性能。
We study the performance of decentralized stochastic gradient descent (DSGD) in a wireless network, where the nodes collaboratively optimize an objective function using their local datasets. Unlike the conventional setting, where the nodes communicate over error-free orthogonal communication links, we assume that transmissions are prone to additive noise and interference.We first consider a point-to-point (P2P) transmission strategy, termed the OAC-P2P scheme, in which the node pairs are scheduled in an orthogonal fashion to minimize interference. Since in the DSGD framework, each node requires a linear combination of the neighboring models at the consensus step, we then propose the OAC-MAC scheme, which utilizes the signal superposition property of the wireless medium to achieve over-the-air computation (OAC). For both schemes, we cast the scheduling problem as a graph coloring problem. We numerically evaluate the performance of these two schemes for the MNIST image classification task under various network conditions. We show that the OAC-MAC scheme attains better convergence performance with a fewer communication rounds.
