Multi-agent systems: coordination does not create general intelligence

A multi-agent system is one in which several autonomous entities act on a shared environment while holding different information, different interests, or both. That is the definition given by Yoav Shoham and Kevin Leyton-Brown in the field's reference textbook, and it comes with a warning worth not skipping: the authors expressly decline to give a precise definition, because many mutually inconsistent ones have been offered, and because even the simpler question —what is a single agent— has resisted a definitive answer. That the canonical book opens by admitting this says more about the terrain than any summary could.

What turns a set of programs into a multi-agent system

The word is used today for almost anything, so the threshold is worth fixing. Three things must hold at once: each entity has its own goal, it observes only part of the environment, and it decides on that basis. If a central orchestrator decides every step and the other components merely execute what they are told, that is a pipeline with stages, not a system of agents. The distinction is not academic purism: it changes what can fail and how you evaluate it.

From there comes the most widespread misunderstanding of the moment. Products that chain model calls with tools —search, run code, write a file— are marketed as «agents», and in commercial terms they are. But as long as a single controller sets the script, the problems that define the technical field never appear: nobody is optimising a goal of their own against everyone else. Saying so is not disdain for those products, which do real work; it is refusing to import into their terrain results that were demonstrated in another.

Cooperate, compete, and the mixture

Multi-agent systems are classified by how the goals relate. In cooperative ones everybody shares a goal and the problem is dividing the work. In competitive ones, what one gains another loses. And there are mixed ones, the closest to the world: teams that collaborate internally and compete with other teams.

That difference is not a label, it is a different evaluation method. In a competitive environment there is no such thing as «the system's score»: one agent's reward may be exactly another's penalty, and a number that goes up without saying for whom means nothing. This is why serious work in the field declares the type of environment before the result — the paper that introduced MADDPG carries it in its title: mixed cooperative-competitive environments.

The problem that makes everything else hard

Here is the technical heart, and it is an idea that needs no mathematics. When a single agent learns, the environment stays put: the rules of the world do not change while it improves. In a multi-agent system, each agent's environment includes the others, and the others are learning too. The ground moves under everyone's feet at once.

This has a name —non-stationarity— and measured consequences. The authors of MADDPG state it at the outset: in the multi-agent case, traditional reinforcement learning runs into an inherent non-stationarity of the environment that challenges Q-learning, while policy-gradient methods suffer a variance that grows with the number of agents. That is the underlying reason why many brilliant single-agent results do not carry over: it is not that the algorithm got worse, it is that the problem stopped being the same one.

And there is a second, quieter difficulty: knowing whom to credit. If a team wins, which agent brought it about? Without answering that, everyone gets the same signal whatever they do. The COMA method attacks exactly that point with an elegant idea: a counterfactual baseline that marginalises out one agent's action while keeping the others' fixed — that is, asking what would have happened had that agent done something else and everything else stayed the same. It is the question to put to any flashy demonstration of coordination.

What it costs to decide without seeing everything

A realistic multi-agent system adds one more constraint: each agent sees only a slice of the world's state. And that constraint is not an engineering nuisance, it has a proven price. Daniel Bernstein, Shlomo Zilberstein and Neil Immerman generalised Markov decision processes —with and without full observability— to decentralised control, and proved that even for a small number of agents the corresponding finite-horizon problems are complete for nondeterministic exponential time. Their own conclusion is that this illustrates a fundamental difference between centralised and decentralised control of Markov processes: unlike single-agent problems, theirs provably admit no polynomial-time algorithms.

Translated into practice: coordinating without a central brain that sees everything is not «the same thing, distributed». It is a different problem, and a costlier one. When someone presents a system of agents that decides on its own, that is the magnitude against which the demonstration should be measured.

What shows up when you let them compete

The fascinating part of the field is that interaction produces things nobody programmed. In OpenAI's hide-and-seek experiment, Bowen Baker and colleagues set two teams to play with a simple objective —hide and seek— and standard reinforcement learning at scale. What they found was an autocurriculum: each team's strategy creates the pressure that pushes the other to adapt, and so on. They document six distinct emergent phases, several of which require sophisticated tool use and coordination; among them, agents that learn to build shelters out of movable boxes.

That is worth reading precisely, because it is where headlines slip in. It does not show that competition generates general intelligence: it shows that in that environment, with those objects and at that compute scale, mutual pressure produced six rounds of new strategies. The result is real and bounded, and the bounded part is as informative as the other.

The capability the reader takes away

Faced with any system presented as multi-agent, four questions separate the technical from the commercial. Does each entity have its own goal and partial observation, or is there a controller deciding everything? If the latter, it is a pipeline, and the field's results do not apply. Cooperative, competitive or mixed? Because what «it works» means, and for whom, depends on that. How is credit assigned? If nobody knows which agent contributed what, the improvement may come from one while the others get in the way. And what does each agent see? — because deciding on partial information without a central coordinator is, provably, far costlier than doing it with a brain that sees everything.

It is also worth placing the taxonomy of human-AI collaboration that this entry cites: it is work on how people and AI systems divide tasks, not on agents coordinating with each other. It may interest readers coming from the product side, but the technical canon for this subject is the other five references in this piece: Shoham and Leyton-Brown's textbook for the foundations, Bernstein and colleagues for the cost of deciding without seeing everything, MADDPG for non-stationarity, COMA for credit assignment, and OpenAI's hide-and-seek for what emerges when they are allowed to compete.

That list of questions depends on no tool and outlives the next generation of products with «agents» on the box. What separates a multi-agent system from a chain of calls is not the sophistication of the model: it is who decides, with what information, and against whom.