Open Source
An Axiomatic Framework for Reinforcement Learning in Open Multi-Agent Systems
arXiv cs.MA0 views
arXiv:2611.03301v3 Announce Type: replace-cross Abstract: Open multi-agent systems, where only a subset of agents are controllable, are central to real-world domains such as smart grids, yet lack a principled foundation for reinforcement learning algorithm design. We study this setting through n-agent ad hoc teamwork (NAHT), a recent framework for learning in open multi-agent environments. We propose an axiomatic framework that derives value function learning from the Shapley axioms (Efficiency, Symmetry and Linearity), treating them as structural constraints rather than heuristic design choices. This formulation builds on a novel NAHT decomposition, which characterises the fundamental difference between open and fully controllable multi-agent systems, and establishes a cooperative game-theoretic foundation for learning. We further show that enforcing these axioms on learning individual value functions recovers the Shapley value for Dec-POMDPs, providing a principled bridge between reinforcement learning and classical cooperative game theory. Building on this framework, we introduce Shapley Machines and Banzhaf Machines, two generic algorithmic templates that can be instantiated with standard reinforcement learning methods. When applied to strong base algorithms such as IPPO and POAM, our methods consistently improve learning efficiency and generalisation to unseen agent types and conventions across NAHT benchmark tasks. The code of experiments is published on https://github.com/hsvgbkhgbv/axiomatic-rl-naht.
Source:https://arxiv.org/abs/2611.03301PreviousMMAO: Mechanism and Stability Analysis of Metabolic Closed-Loop MetaheuristicsNextWhen to Move: Online Repositioning of Movable Antenna Arrays Under a Long-Term Actuation Energy Budget
Related posts
Do Underwater Image Quality Metrics Agree with Human Perception? A Benchmark with Human Choices, Simple Edits and Generative Drift
0arXiv eess.IV
X-Strata: Separating Aneurysms and Overlapping Vessels into Layers from 2D X-ray Angiography
0arXiv eess.IV
What Should World Models Forget? Stratified Retention for Continual Adaptation
0arXiv eess.SP
uhd-rfsoc: Bringing Off-the-Shelf RFSoC Boards to Unmodified UHD Applications
0arXiv eess.SP
From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
0arXiv cs.MA
A Taxonomy of Real Faults in Hybrid Quantum-Classical Architectures
0arXiv cs.SE