Survey: How Graph Neural Networks Are Revolutionizing Communication in Multi-Agent Reinforcement Learning

Introduction: The Core Challenge of Multi-Agent Collaboration

In the frontier of artificial intelligence research, teaching multiple agents to collaborate in complex environments remains an extremely challenging task. Recently, a new survey paper (arXiv: 2604.25972) was published that systematically reviews research progress in multi-agent deep reinforcement learning (MARL) with graph neural network (GNN)-based communication mechanisms, providing a clear academic panorama for the field.

The core challenge of multi-agent reinforcement learning lies in the fact that when multiple agents simultaneously learn and make decisions in an environment, each agent effectively faces a "non-stationary" environment — because other agents are also constantly adjusting their strategies. This dynamic game-theoretic nature makes it difficult to directly transfer traditional single-agent reinforcement learning methods. Introducing communication mechanisms that allow agents to share information and coordinate actions is considered one of the key approaches to solving this dilemma.

Core Methods: GNN-Driven Agent Communication

Interaction Graph Modeling

The survey focuses on a particular class of communication methods — communication architectures based on interaction graphs. In these methods, relationships among agents are modeled as graph structures, where nodes represent agents and edges represent communication links between them. This graph structure is naturally suited for describing the complex interaction topologies in multi-agent systems.

Unique Advantages of GNNs

Graph neural networks play a central role in these architectures. Unlike traditional fully-connected or broadcast-style communication, GNNs offer the following key advantages:

• Local information aggregation: Each agent can aggregate information from neighboring nodes through GNN message-passing mechanisms, progressively enriching its internal representations
• Topology adaptability: GNNs can naturally handle dynamically changing communication topologies and adapt to varying numbers of agents
• Scalability: Compared to global communication schemes, GNN-based methods offer better computational scalability as the number of agents grows
• Structured representation learning: GNNs enable agents to learn structured relational representations rather than simple information concatenation

Information Exchange and Representation Enhancement

The paper notes that the core idea behind these methods is that agents learn "how to communicate" and "what to communicate" through GNNs. In each interaction round, agents encode their observations and transmit them to neighbors through the graph structure while receiving neighbors' information. Multi-layer message passing in GNNs allows information to diffuse across a wider range, enabling each agent to gain global awareness beyond its own local observations.

Technical Analysis: Method Classification and Comparison

Design Dimensions of Communication Structures

From the survey's framework, GNN-based MARL communication methods can be classified along the following dimensions:

1. Graph Structure Determination
- Predefined graphs: Communication topology set in advance based on agents' physical distances or domain knowledge
- Learned graphs: Communication graphs dynamically generated through attention mechanisms or other learnable modules
- Hybrid approaches: Combining prior knowledge with learning capabilities

2. Message Content Formats
- Encodings of raw observations
- Abstract expressions of intentions or goals
- Gradient information from policies or value functions

3. GNN Architecture Choices
- Graph Convolutional Network (GCN)-based approaches
- Graph Attention Network (GAT)-based approaches
- Message Passing Neural Network (MPNN)-based approaches

Comparison with Traditional Communication Methods

Compared to earlier fully-connected network-based communication methods (such as CommNet and TarMAC), GNN methods demonstrate clear advantages in the following scenarios:

• Large-scale agent systems: When the number of agents reaches dozens or even hundreds, GNN's local aggregation mechanism significantly reduces communication and computational overhead
• Heterogeneous agent scenarios: GNNs can flexibly handle differentiated communication needs among different types of agents
• Partially observable environments: Through multi-hop message passing, GNNs help agents overcome the limitations of local observations

Application Scenarios and Potential

GNN communication-based MARL methods have demonstrated application potential in multiple domains:

• Autonomous vehicle fleet coordination: Multiple autonomous vehicles share road condition information through communication graphs to achieve cooperative driving
• Multi-robot collaboration: Task allocation and path planning in scenarios such as warehouse robots and drone formations
• Network resource scheduling: Collaborative resource allocation among multiple base stations in communication networks
• Multiplayer strategy games: Team collaboration in complex strategy games such as StarCraft

Challenges and Outlook

Despite significant progress in this field, the survey also identifies several key challenges that remain to be addressed:

Communication efficiency: How to achieve efficient information compression and transmission under limited bandwidth while avoiding "communication flooding" remains an open problem.

Dynamic topology adaptation: Further research is needed on maintaining the stability and robustness of GNN communication mechanisms in dynamic scenarios where agents frequently join or leave.

Interpretability: Communication protocols learned by GNNs often lack interpretability. Understanding "what agents are saying" is critical for safety-critical applications.

Sim-to-real transfer: Most research still remains in simulated environments. Deploying GNN communication mechanisms to real physical systems faces practical challenges such as latency and packet loss.

Integration with large language models: With the rise of LLMs in the agent domain, exploring synergies between GNN communication and language model-driven agents may become an important future direction.

Conclusion

This survey provides a systematic review and classification framework for GNN-based communication research in multi-agent reinforcement learning. As graph neural network technology continues to mature and multi-agent application scenarios grow increasingly diverse, GNN-driven agent communication is poised to become one of the core technologies for building large-scale, efficient, and scalable multi-agent systems. For researchers working on MARL, graph learning, and multi-robot systems, this survey is undoubtedly a highly valuable reference.