Multi-agent systems
How to design, own, and ship products where multiple AI agents collaborate — and what breaks when you don't.
Authority Hierarchy
Every multi-agent system needs a chain of command. Without one, agents deadlock waiting for peers to resolve ambiguity — or two agents issue contradicting instructions to a third and the system produces garbage output with no error. The authority hierarchy defines which agent can instruct which. It is a PM-owned document, not an emergent engineering decision.
| Level | Agent | What it does |
|---|---|---|
| L1 | Orchestrator Agent | Receives the task, maintains state, delegates to domain agents, assembles final output. |
| L2 | Domain Agents (Research, Writer, Reviewer) | Executes a scoped subtask, returns structured result to orchestrator. |
| L3 | Tool Workers | Calls a specific API or tool (web search, database query, code runner), returns raw result. |
The hard rule: an agent must receive instructions from exactly one parent. Two domain agents cannot both delegate to the same worker simultaneously — the orchestrator gates those calls sequentially, or you get race conditions in the output.
Warning: Peer-to-peer delegation — two domain agents instructing each other — is the most common hierarchy violation in MAS design. It creates circular dependency chains that are nearly impossible to debug in production.
Context vs. Stateless Agents
The distinction between context-carrying and stateless agents is the most consequential design decision in a MAS — and most PMs leave it to engineers. A context agent maintains the running task state across every step. A stateless agent receives a self-contained instruction packet, executes exactly that, and returns a result with no memory of prior calls.
| Context Agent | Stateless Agent | |
|---|---|---|
| Memory | Maintains full task history | Only what's in the handoff payload |
| Cost | Higher — more tokens per call | Lower — minimal context per call |
| Debug | Hard — failure depends on accumulated state | Easy — reproduce by replaying the payload |
| Blast radius | Entire task if it fails | Single subtask only |
| Default choice | Orchestrator only | All subagents |
Rule of thumb: Default every subagent to stateless. The orchestrator pays the cost of knowing everything so the subagents don't have to. Stateless subagents are also easier to swap — if your Research agent underperforms, you can replace it without touching anything else in the pipeline.
Agent Scoping
An unscoped agent is a liability. Given an out-of-scope request, a specialist agent will not return an error — it will try to answer anyway, hallucinate, and pass the result upstream with no flag that it was operating outside its domain. Scope must be explicit, not implicit.
Warning: "The agent will just know what it can't do" is not a scoping strategy. Models will attempt tasks they shouldn't. Scope rejection must be an explicit, enforced behavior — a structured response like { status: "out_of_scope", reason: "..." } — so the orchestrator handles it without treating it as a crash.
The Agent Spec
The Agent Spec is the PM's primary deliverable in MAS — what you write before engineering designs anything. Without a spec, the system has no reference for correct behavior. When an agent misbehaves in production, you need to compare what it did against what it was supposed to do. If the spec doesn't exist, that comparison is impossible.
Agent Name: [e.g., Research Agent] Role: [One sentence — what is its single job] Authority Level: [L1 Orchestrator / L2 Domain / L3 Tool Worker] Parent Agent: [Which agent delegates to this one] Tools Allowed: [Explicit list — e.g., web_search, read_file] Input Schema: [Fields the handoff payload must contain] Output Schema: [Fields the result must contain] Max Steps: [Hard cap on tool calls per invocation] Failure Behavior:[What it returns on error — structured, not an exception] Escalation Path: [Which agent or human receives the escalation signal] Scope Rejection: [What inputs trigger a rejection and what the signal looks like]
Good practice: Treat the Agent Spec like an API contract. Engineers don't change the input/output schema without a versioning discussion. PMs don't change the scope without updating the spec. Both treat the spec as the source of truth.
Choosing a Pattern
Pattern selection is a product decision with direct consequences for cost, speed, auditability, and output quality. The PM owns this choice; engineering implements it.
A planning agent receives the task, maintains context, breaks work into subtasks, and delegates to stateless specialist agents. The orchestrator holds state; subagents execute and return results.
Agents run in a fixed chain — output of Agent A feeds input of Agent B. No parallelism, but fully auditable. Every step is logged, reviewable, and reproducible.
Multiple agents execute independent subtasks simultaneously. A synthesizer agent merges and deduplicates results. Fastest pattern for work that can be split without dependencies.
Agent A produces output; Agent B critiques against a quality rubric; Agent A revises. Loop continues until a quality threshold is met or a max iteration count is hit.
The tradeoff: Sequential gives you an audit trail but no speed. Parallel gives you speed but adds a synthesizer failure point. Peer review improves quality but multiplies cost.
You're the PM for a content pipeline with an Orchestrator, a Research Agent, and a Writer Agent. Write the Agent Spec for the Research Agent using the template above.
A colleague says 'the agents will figure out who delegates to whom at runtime.' Write a one-paragraph rebuttal explaining why authority hierarchy must be defined in the spec — with a concrete example of what breaks when it isn't.
Your Research Agent is returning out-of-scope results into the pipeline. You don't have a scope rejection mechanism. Design one — what signal does it return and how does the orchestrator handle it?
Every agent has exactly one parent. Peer-to-peer delegation creates circular dependencies that are undebuggable in production.
The orchestrator holds state so subagents don't have to — this contains blast radius and makes individual agents swappable.
An agent that returns a structured "out of scope" signal is safer than one that tries to answer and hallucinates.
If the spec doesn't exist before the orchestration layer is designed, you'll pay to rewrite both when behavior needs to change.
Sequential pipelines are auditable. Parallel fan-out is fast. Peer review loops improve quality but multiply cost. These are product decisions with business consequences — not engineering preferences.
What's in the current context window — system prompt, conversation history, tool results accumulated so far. Zero infrastructure required.
The PM defines what must be in context at each step — in the agent spec. "Pass everything" is not a schema.
Structured state persisted outside the model — relational DB, key-value store, or vector store. Multiple agents can read from and write to it. Survives session boundaries.
The PM defines what fields are persisted, which agents can write vs. read, and the retention policy — before engineering provisions the database.
The structured data packet an orchestrator sends to a subagent at delegation time. Lives only for the duration of that one subagent call — ephemeral, not persisted.
Handoff payload schemas are PM-owned artifacts in the agent spec. Engineers implement them; PMs define the fields.
Warning: Choosing "in-context for everything" because it requires no infrastructure is a common trap. When task history grows to 50+ steps across a pipeline, in-context costs compound fast — and there is no audit trail when something goes wrong.
The handoff payload is the most underspecified artifact in MAS design. PMs write agent specs with input/output schemas, then leave the handoff content to engineering as an implementation detail. That is wrong — the payload determines what the subagent can do.
{
"instruction": "Research pricing strategies for SaaS companies"
}No context, no format, no constraints. Agent retrieves general information and wastes tokens.
{
"task_goal": "Find 3 pricing examples from B2B SaaS companies ARR $10M–$50M",
"user_context": "PM at a Series B startup benchmarking mid-market peers",
"prior_steps_summary": "Competitor analysis complete. [A, B, C] identified.",
"required_output_format": {
"examples": [{ "company": "", "model": "", "price_points": [], "source_url": "" }]
},
"max_tokens": 1500,
"scope_constraints": "Only B2B SaaS. No freemium. No IPO'd companies."
}Structured, scoped, constrained. The agent can act immediately and return a usable result.
PM rule: Handoff schemas are versioned artifacts. Treat them like API contracts — version them, don't silently mutate them. If the Research Agent's output format changes, the orchestrator's parsing logic must change too.
Summarization Gates
After N steps, a summarizer agent compresses accumulated history into a fixed-size summary, which replaces the raw history in the orchestrator context.
PM decides: what is N? What must be preserved verbatim? What can be lossy-compressed?
The summarizer can drop details that matter later. Define non-compressible data in the spec.
Long-running tasks where the orchestrator must maintain continuity but raw history is too large.
Selective Passing
The orchestrator maintains a structured task state and passes only the relevant subset to each subagent — not the full accumulated history.
PM decides: what goes in the task state object? What does each subagent actually need from prior steps?
The orchestrator's relevance judgment can be wrong. Important context gets dropped silently.
Pipelines where subtasks are cleanly separated with low cross-dependencies.
External State Offloading
Completed step results are written to an external store. The orchestrator holds a reference (ID, key), not the full content. Subagents retrieve what they need directly.
PM decides: what store? What schema? Who has read/write access? What is the retention policy?
More infrastructure, more latency (50–200ms per retrieval), more failure modes to manage.
High-scale pipelines where context limits are a recurring constraint, not a one-off.
Warning: Context strategy is a launch criterion, not a post-launch optimization. If your pipeline hits context limits at step 15 and you haven't specced a compression strategy, you will be debugging it in production on a live task.
Design the handoff payload schema for a Writer Agent receiving output from a Research Agent. What 6 fields would you include, and why?
Your orchestrator is hitting context limits at step 12 of a 20-step pipeline. Walk through the three compression strategies and pick the one that fits best for a customer support pipeline. Justify the choice.
A subagent needs to know what steps have already been completed, but sharing the full context is too expensive. Design an external state schema that solves this for a 5-agent research pipeline.
If you haven't defined the fields, the engineers will guess — and the subagent will get the wrong context.
Every token in context is charged on every call. Long task histories compound fast across a pipeline.
Choosing the wrong strategy in production costs a full re-architecture, not a hotfix.
Which agents can write, which can only read, and the retention policy — these are product decisions, not infrastructure decisions.
Cost Multiplication
A 4-agent pipeline does not cost 4× a single-agent call. It can cost 10–20× if agents call each other recursively or if an orchestrator fans out to multiple subagents per task.
Orchestrator (2K tokens) → 3 parallel Research Agents (3K each) → Synthesizer (4K) = ~15K tokens per task. At ~$3/MTok input (Claude Sonnet 4.6, Jun 2026), that's $0.045/task. At 200K tasks/month: $9,000/month in input tokens alone — before output tokens at ~$15/MTok.
Pricing approximate as of Jun 2026. Verify current model pricing before building cost models.
Use Flash-class models for retrieval and formatting steps. Reserve frontier models for judgment steps only. A pipeline where only 1 of 4 agents uses a frontier model costs 60–70% less.
Cap parallel subagent calls per orchestrator invocation. Test whether 3 parallel Research Agents produce 3× better output before assuming the cost is justified.
Hard cap on total tool calls per task. An agent without a step budget will retry indefinitely on ambiguous inputs — burning tokens on a task that should have escalated.
Warning: "We'll optimize cost after launch" is the most expensive sentence in MAS product development. Model tier routing and fan-out limits must be in the PRD before the first API call.
Failure Cascades
In a sequential pipeline, a failure in Agent 2 doesn't just fail that step — it sends corrupted output to Agent 3, which processes it as valid, passes a confidently wrong result to Agent 4, and delivers it to the user. No error is surfaced. The system "completes" the task.
Research Agent returns a hallucinated competitor pricing figure → Writer Agent cites it with full confidence → Reviewer Agent checks formatting and tone, not factual accuracy → User receives fabricated data presented as fact.
Which steps halt on failure? Which continue with a degradation flag? A Research Agent returning low-confidence results should stop an analysis pipeline. A Formatter Agent failing should not block delivery.
Each agent's output schema includes a confidence score. The orchestrator checks before passing to the next step. Below threshold → escalate, not pass.
High-risk steps — anything that writes to a database, sends a communication, or produces output that will be acted upon — must have a human review gate before the next step executes.
Note: Fail-fast vs. fail-safe is a product decision, not an engineering default. If the PRD doesn't specify it, every engineer on the team will make a different call for each step they own.
Observability
When a 4-agent pipeline produces a wrong result, the first question is: which agent caused it? Without per-agent trace logs, the answer is "we don't know" — and that means debugging by intuition in production.
| Field | Why it matters |
|---|---|
| Agent ID + version | Which agent, and which version of its prompt/config |
| Timestamp (start + end) | Latency per step; detect slowdowns before they cascade |
| Input received | Exact payload — enables replay for debugging |
| Output produced | Exact result — compare against spec's output schema |
| Tools called | Which tools, in what order, with what arguments |
| Step count | Detect agents approaching their max step cap |
| Tokens used (input + output) | Per-agent cost attribution |
| Confidence score | Whether the agent flagged uncertainty |
| Escalation reason | If escalated, why — must be structured, not free text |
Warning: Observability is a launch criterion, not a future sprint. A MAS without per-agent trace logging is a black box in production. The incident playbook — who investigates a P0, what trace to pull first, what defines P0 vs. P1 — must also exist before launch, not be written the day of an incident.
Trust Boundaries
Every connection between agents in a MAS is a trust boundary. The orchestrator trusts the subagent's output. The subagent trusts the orchestrator's payload. Neither validates the other by default — unless you design it in.
Does the orchestrator validate a subagent's output against the expected schema before using it? Answer: yes, always. An agent returning malformed output should trigger a structured error, not a downstream parse failure. Define "valid" for each agent's output in the spec.
Does a subagent inherit the orchestrator's tool permissions? Answer: no. Each agent has its own permission scope, defined in its spec. A Research Agent that can only read web sources should not inherit a DB write permission just because the orchestrator has one. Least-privilege per agent.
Any agent that processes user-supplied text is a prompt injection surface. A malicious user can craft input that instructs the agent to ignore its system prompt or call a tool it shouldn't. Every agent that touches user input must be flagged in the agent spec with injection-hardening instructions.
Warning: Permission inheritance is the most common trust boundary violation in early MAS deployments. Engineers give the orchestrator broad permissions for convenience during development, subagents inherit them by default, and production launches with subagents that have far more access than they should.
When four agents touch a task and something goes wrong, the default answer to "who owns this incident?" is: no one. Each agent in the pipeline assumed another agent validated the output. None did. This is not an engineering failure — it is a missing PM artifact. Before any multi-agent system launches, the PRD must include an agent responsibility matrix: which agent is authoritative for which decision, which human it escalates to, and what constitutes a P0 versus a P1 in this pipeline.
What Goes in the MAS PRD
A standard agent PRD documents what the agent does, its success metrics, and its rollout plan. A MAS PRD needs 8 additional items that a single-agent PRD doesn't require:
Your 4-agent pipeline runs 50K tasks/month. Orchestrator uses Claude Sonnet, all 3 subagents use Flash-Lite. Estimate the monthly input token cost. What happens if you move the Writer Agent to Sonnet?
Design the fail-fast vs. fail-safe rules for a 4-step pipeline: Research → Draft → Review → Publish. Which steps halt on failure? Which continue with a flag? What is the confidence threshold for each?
Write the trust boundary documentation for a Summarizer Agent that processes user-uploaded documents. Cover: output validation spec, permission scope (not inherited), and prompt injection risk.
Fan-out and recursive calls compound fast. Model tier routing must be in the PRD before the first API call.
A pipeline without confidence thresholds and fail-fast decisions will deliver confident wrong answers — not errors.
You cannot diagnose a 4-agent system without per-agent trace logs. "We'll add it later" means debugging blind.
Audit every agent's effective permissions before launch. No subagent should inherit orchestrator permissions by default.
In a multi-agent pipeline, every agent assumes another caught the error. No one did. An agent responsibility matrix — which agent owns which decision, who it escalates to — must be in the PRD before engineering starts.
Avoid when: When task needs parallel execution a single agent cannot provide, or when specialist expertise genuinely requires different models or permissions per subtask.
Avoid when: When a single well-prompted agent can do the job — a multi-agent system adds coordination overhead, latency, and debugging complexity with no corresponding gain.
PM rule: If a well-prompted single agent can complete the task reliably, that is the correct architecture.
The task has multiple phases and the output of early phases must persist into later phases beyond what a single context window can hold. Canonical case: a 50-page document analysis requiring chained sessions.
Two or more independent subtasks can run simultaneously, and the time savings from parallelism exceeds coordination overhead. Below 60% wall-clock reduction, a sequential single-agent approach is usually simpler and cheaper.
One subtask needs database write access; another needs only read; a third needs web access but no database. If permissions diverge, scoped specialist agents are safer than one agent with all permissions.
You've tested it. A Code Review Agent with a linting-focused system prompt produces 40% fewer false positives than a general agent asked to "review this code." Quality improvement is measured, not assumed — this signal requires an A/B test.
The task involves investigation that spawns sub-investigations, with a defined max depth. At max depth, the agent returns a partial result rather than spawning further. Unbounded recursion is a runaway cost and reliability risk.
Note: If you're counting how many signals apply, the answer you want is at least two. One signal in isolation rarely justifies MAS complexity. The strongest justifications combine Signals 1+4 (context limit + quality improvement) or 2+3 (parallelism + permission separation).
The fix is better prompt engineering, not more agents. A 3,000-token system prompt with poor structure is not a MAS problem. Refactor the prompt — agents don't fix bad prompts; they amplify their failures.
In a 4-agent pipeline where each agent has 90% reliability, end-to-end completion is 0.9⁴ = 65.6%. Adding coordination to a failing baseline makes it worse. Fix the single agent first.
If you cannot write the output schema for each agent, you cannot build the system. Inability to spec the handoff means you don't understand what each step does — and a MAS built on undefined handoffs will be redesigned after the first production failure.
Multi-agent systems require an org that understands how agents fail — context limits, hallucination patterns, tool call errors. If your team is still learning these on a single agent, a MAS will surface them all simultaneously at a scale you cannot diagnose.
Parallel agents reduce wall-clock time for independent tasks only. Sequential coordination adds latency. If your user-facing requirement is sub-second and your task has sequential steps, a 4-agent pipeline will be slower than a single-agent call.
Warning: "We need agents for this" is often a solution looking for a problem. Before designing a MAS, write down which go-signals apply to your specific task. If you can't name two, build a better single-agent system instead.
Run this checklist on any candidate task before committing to a MAS design. Each question has a clear pass/fail outcome.
Can the task be split into subtasks with clearly defined outputs?
Write the output schema for each subtask. If you can't write the schema, you can't build an agent for it.
Are the subtasks independent enough that a subagent can be briefed without the full task context?
Write the minimum payload the subagent needs to execute. If that requires sending the full task history to every subagent, the tasks aren't decomposed — they're duplicated.
Does at least one subtask benefit from a different model, tool set, or permission scope?
If every subtask could be done by the same model with the same tools and permissions, evaluate your justification before building.
Is there a human review point before any irreversible action executes?
Identify every irreversible action in the pipeline. Confirm there is a human review gate before it executes. No exceptions.
Gate: A task that passes all 4 questions is a strong MAS candidate. A task that fails Question 1 or 4 should not be built as a MAS until those are resolved — no exceptions.
Narrow scope — customer service only, no open-ended tasks. Clear success metrics defined before launch. Measurable ROI from Day 1. The agent spec explicitly listed what the system could not do, which was as important as what it could.
Agent mode is scoped strictly to code within the user's repository. The feedback loop is immediate — the user sees every change. The user retains full control and can revert instantly. Trust radius was designed in, not added after.
Domain-constrained to legal research only. Human-in-the-loop for every final output before it reaches a client. Deliberately chose reliability over capability breadth — the product does less than it theoretically could, by design.
Promised full autonomy on complex, open-ended tasks. In production, more than 70% of tasks still require human correction. Overpromising autonomy at launch destroys user trust in a way that is very difficult to recover from.
The pattern: Every working system shares narrow scope, a defined trust radius, and success metrics set before engineering starts. The cautionary tales share one pattern — autonomy claims that outran reliable delivery. That gap is a PM specification failure, not an engineering failure.
You're evaluating this task: 'Analyze a customer's contract and generate a renewal proposal.' Run the 4-question Decomposability Test on it. For each question: answer it, explain your reasoning, and give the pass/fail verdict.
Your team wants to build a 3-agent pipeline for a task that currently has a 65% completion rate with a single agent. Write the argument against building it now, and what the single agent needs to achieve before MAS is justified.
A colleague proposes using 4 parallel agents to reduce response time from 8 seconds to 2 seconds. What's the first question you ask? Walk through which go-signals apply and which anti-patterns might be present.
One signal in isolation rarely justifies MAS complexity. Look for at least two before committing.
A sub-70% completion rate on a single agent predicts a worse outcome in a multi-agent pipeline — not a better one.
Parallel agents reduce wall-clock time for independent tasks only. Sequential coordination adds latency. If speed is the bottleneck, MAS is the wrong tool.
Failing Question 4 (no human review before irreversible actions) is a hard stop. Don't launch without it.
Klarna, Cursor, and Harvey all work because the agent's scope was explicitly bounded before engineering started — what it can do, and what it cannot. Breadth is the enemy of reliability.