论文标题
多代理强化学习中的普遍表达性沟通
Universally Expressive Communication in Multi-Agent Reinforcement Learning
论文作者
论文摘要
允许代理商通过通信共享信息对于解决多代理增强学习中的复杂任务至关重要。在这项工作中，我们考虑了给定通信协议是否可以表达任意政策的问题。通过观察许多现有协议可以看作是图神经网络（GNN）的实例，我们证明了联合动作选择与节点标记的等效性。通过证明其表达能力的标准GNN方法，我们借鉴了现有的GNN文献，并考虑使用以下方式增强代理观测值：（1）独特的代理ID和（2）随机噪声。我们提供了关于这些方法如何产生普遍表达性交流的理论分析，并证明它们能够针对相同代理的任意行动集。从经验上讲，这些扩展被发现可以改善需要表达性交流的任务的性能，而通常发现最佳通信协议是任务依赖性的。
Allowing agents to share information through communication is crucial for solving complex tasks in multi-agent reinforcement learning. In this work, we consider the question of whether a given communication protocol can express an arbitrary policy. By observing that many existing protocols can be viewed as instances of graph neural networks (GNNs), we demonstrate the equivalence of joint action selection to node labelling. With standard GNN approaches provably limited in their expressive capacity, we draw from existing GNN literature and consider augmenting agent observations with: (1) unique agent IDs and (2) random noise. We provide a theoretical analysis as to how these approaches yield universally expressive communication, and also prove them capable of targeting arbitrary sets of actions for identical agents. Empirically, these augmentations are found to improve performance on tasks where expressive communication is required, whilst, in general, the optimal communication protocol is found to be task-dependent.
