AI Agent Orchestration: Coordinating Specialized Agents at Enterprise Scale

At enterprise scale, AI agent orchestration is a memory problem before it is a routing problem. Deciding which agent runs next is the easy half. The hard half is making sure the security agent’s Tuesday discovery reaches the QA agent’s Wednesday test run, without letting what Legal knows leak into what Sales can recall. eToro runs 300+ specialized agents against one governed memory layer holding 26,500+ memories at 23 ms p50 search, and the thing that made it work was not a smarter router.

Here is a number that frames the gap. We read the primary memory or state documentation page for seven agent frameworks and orchestration SDKs. Five of the seven describe memory that more than one agent or process can reach. One of the seven documents a per-agent access-control mechanism. None of the seven uses the phrase “audit log” or “audit trail” anywhere on that page, and none documents per-agent trust tiers.

That is the shape of the problem. This blog covers the four control patterns worth knowing, why they degrade past roughly a dozen agents, the specific mechanism that fixes it, four runnable reference implementations you can clone this afternoon, and what the whole thing costs to operate.

What is AI agent orchestration?

AI agent orchestration is the layer that decides which specialized agent acts, in what order, with what context, and what happens to the result. A single agent with tools is not orchestration. Orchestration starts the moment two agents need to agree on something.

The distinction that matters in practice: a workflow engine coordinates steps, and an orchestrator coordinates agents. A step is deterministic and you wrote it. An agent decides its own next move, which means the orchestrator has to handle non-determinism, partial failure, and the fact that two agents can reach opposite conclusions from the same inputs.

The four control patterns

Almost every production system is one of these four, or a composition of them.

Anthropic’s engineering write-up on its multi-agent research system is the clearest public description of the hierarchical pattern, including the detail that the lead agent saves its plan to memory so a context overflow does not destroy it. That single detail is the whole argument of this article in miniature: the moment orchestration gets serious, state moves out of the prompt.

The first three patterns are the ones frameworks make easy to build. The fourth is the one enterprise fleets converge on, and it is the one that fails hardest without a governance layer.

Why does orchestration break at enterprise scale?

It breaks in four specific ways, and all four are symptoms of the same root cause: the orchestrator is carrying knowledge in the prompt instead of in a store the fleet shares.

The coordination tax compounds with agent count

In a naive multi-agent loop, input tokens scale with agents multiplied by history length multiplied by redundancy. None of those three multipliers is capability. Each agent re-sends its own growing transcript every turn. Handoffs forward whole conversations rather than the relevant slice. Shared conventions get copied into every window. And the multiplier unique to fleets: one agent solves a problem, a sibling hits the same wall two hours later and pays full price to solve it again.

A bigger context window does not remove this. It raises the ceiling on a bill that should not exist. Caura’s own measurement of the effect is a 96.6% to 98.2% reduction in tokens versus carrying full context, measured on the LoCoMo and LongMemEval benchmarks. The mechanism is covered in more depth in their breakdown of the token tax of multi-agent systems.

Stale facts survive, and nothing tells the fleet which one is current

This is the failure mode that vector stores handle worst. When a fact changes in the world, a plain vector store adds the new version alongside the old one. Nothing supersedes anything.

There is a runnable demonstration of exactly this. The long-run fleet reference implementation simulates three agents (sourcing, verification, synthesis) running daily for 14 days. A competitor’s pricing page reads $299/month for Days 1 through 8. On Day 9 it changes to $349/month. In a raw vector store, the pool now holds eight reinforced memories saying $299 and one saying $349, and the synthesis agent has no way to rank truth over repetition.

In the repo’s run, the Day 9 write triggers async contradiction detection, which marks all eight $299 entries outdated. On Day 10 the synthesis agent’s recall returns one result, and the brief reports how many were filtered. The stale entries are suppressed by status, not by a prompt instruction telling the agent to ignore old data.

Prompt-level separation is not access control

This is the one that stops enterprise deployments in security review. Telling an agent “do not mention compliance data” does not prevent the data from being retrieved. It passes through recall, sits in context, and the model can still surface it under pressure.

The cross-fleet governance repo demonstrates the alternative with three agents (Sales, Legal, Admin) over one memory backend. When the sales agent writes a $50k Acme deal to fleet-sales, the legal agent asking “what do you know about Acme Corp?” gets zero results. Not a filtered result. Zero, because the recall call resolves to a WHERE fleet_id IN (‘fleet-legal’, ‘fleet-org-shared’) predicate that runs before the hybrid search. Rows outside the declared fleets are never loaded, never scored, never ranked.

The admin agent holds all three fleets, which makes it the only one that sees both the pipeline entry and the legal hold, and therefore the only one that can surface the conflict between them.

The repo is honest about the limits of this in an open-source local deploy: fleet isolation there is a query-layer contract that depends on agents declaring their fleet_ids correctly. For isolation that cannot be bypassed by changing a parameter, the documented pattern is separate tenants per domain, with the admin agent doing explicit fan-out recall and merging results with source labels.

Every new agent starts from zero

Add the fortieth agent to a fleet and it arrives with excellent reasoning and no idea that your payments team never retries a charge without an idempotency key, or that “the staging cluster” means eu-west-1. The usual fixes do not hold. System prompts balloon and go stale. Plain retrieval is ungoverned, so every agent sees the same undifferentiated blob. Fine-tuning is wrong the moment a policy changes.

Caura’s write-up on the agent cold-start problem splits this into two questions with two different answers: what does the org already know (discretionary, solved by scoped ingestion) and how must the org require the agent to behave (mandatory, solved by policy rules read at session start).

What does the memory layer have to do with orchestration?

It is the coordination substrate. Once agents read and write a shared, governed store, the orchestrator stops having to carry knowledge between them, and coordination becomes a property of the data rather than a property of the control flow.

Concretely, four mechanisms do the work.

Scope stamped at write time

Every memory carries a visibility scope the moment it is written: scope_agent (private to the writing agent, enforced as a per-row server-side ACL), scope_team (readable across the fleet), or scope_org (cross-fleet, permissioned). Scope is a structural field, not a filter someone remembers to apply later. A support-fleet recall cannot surface an HR-fleet memory because the row was never in the candidate set.

Trust tiers on top of scope

Scope decides how far a fact travels. Trust decides which agents can travel that far. Caura documents four tiers: restricted (0), standard (1, the default for new agents), cross_fleet (2), and admin (3). What those tiers gate is narrower than the names suggest. Reads and writes are open in an agent’s own scope, so even a level-0 agent can recall and write, which makes level 0 a low-privilege label rather than a kill switch. Cross-fleet and fleet-wide reads, meaning stats, list, insights, and fleet-scope evolve, require trust level 2 or higher, and delete requires level 3. Trust is checked server-side on every call, so a level-1 agent attempting a cross-fleet recall gets a 403.

Mandatory rules, delivered deterministically

Retrieval is a search problem, and search can miss. Policy cannot afford a miss. Keystones are mandatory rules fetched deterministically at session start with no semantic ranking and no top_k, merged across tenant, fleet, and agent scope, and framed in the tool description itself as overriding conflicting user instructions. The distinction matters most at turn five of a conversation, when the most emotionally weighted context in the window is the user’s pushback rather than your refund policy. Caura’s post on why system prompts stop holding under pressure walks through the failure with a transcript.

Outcomes feed back into retrieval

Agents report what happened after acting on a recalled memory. Successes reinforce the memories involved. Failures generate a preventive rule, which defaults to the reporting agent’s own private scope. Promoting that rule to fleet scope is a deliberate call that requires trust level 2 or higher, and that promotion is what puts the lesson in front of the next forty agents before they repeat the mistake. Caura calls this the Karpathy Loop, after the hypothesize, experiment, evaluate, persist cycle demonstrated by Andrej Karpathy’s autoresearch project in March 2026. It is the loop a single-agent memory store cannot close, because there is no second agent to close it with. The mechanism sits alongside contradiction detection and the crystallizer in Caura’s governance concepts.

What seven frameworks actually document about cross-agent memory

We read the primary memory or state documentation page for seven agent frameworks and orchestration SDKs on 20 August 2026 and recorded what each page does and does not describe. The result is lopsided, and the lopsidedness is the reason platform teams hit a governance wall in month three rather than in week one.

Method: one page per framework, the page each project presents as its primary memory or state documentation, read on 20 August 2026. Broader platform documentation elsewhere on those sites may cover governance; this measures what a developer finds on the page they land on when they go looking for how memory works.

Two things follow from the table. First, shared memory is close to solved as a plumbing problem: five of seven have somewhere for more than one agent to read from. Second, the governance layer is almost entirely absent from that surface. CrewAI’s private scoping is the only per-agent access control any of the seven documents there, and no framework in the sample documents an audit trail or a trust model on that page.

That gap is not a criticism of these projects. Each was designed around a shape that made sense: one agent, one user, one long conversation, or one graph and one run. It is a description of what a platform team has to build themselves if they orchestrate more than a handful of agents inside a company that has a compliance function.

If you have already hit that wall, Caura exists specifically to be the layer the table is missing: scope on every write, trust checked on every call, an audit trail on every write, delete, and transition, Apache 2.0, and a free tier that holds 10,000 memories with unlimited agents and fleets.

How do you actually wire this up?

Four repositories, all Apache 2.0 and all runnable, cover the three orchestration patterns that carry most enterprise work plus the engine underneath. Clone the one that matches your shape rather than reading all four.

Pattern 1: constraint propagation across a sequential pipeline

caura-build-fleet runs five specialists over one shared memory: Frontend, Performance, SEO, Code Review, and a read-only Manager. Each agent recalls what the previous ones decided before it acts. Performance writes “zero external JavaScript.” SEO recalls that constraint and picks inline JSON-LD instead of a CDN-loaded schema library. Code Review then recalls the whole fleet and cites specific memory IDs in its verdict. Nobody hard-coded that rule into the pipeline. It propagated because agents read each other’s memory.

The detail worth stealing is the per-agent tool allowlist. The Manager agent never receives the write tool definition at all, so its read-only status is enforced at the tool-schema level rather than by instruction. At the end of each run it reports zero writes, which is the isolation proof. Published run timings from the repo: Frontend 18.4s, Performance 21.3s, SEO 33.1s, Code Review 34.0s, Manager 40.7s.

Pattern 2: hard boundaries between departments

caura-cross-fleet-gov is the Sales, Legal, Admin setup described above. Each agent workspace carries three files that the gateway injects at session start: SOUL.md (identity and hard limits), AGENTS.md (authorized fleet_ids, recall protocol, write rules), and IDENTITY.md (the canonical agent_id passed on every tool call, which is what makes per-row ACLs and the audit trail work).

Two runtime behaviors in that repo are worth copying into any long-running fleet.

Bootstrap: before making any memory call, the agent reads its governance skill file, which loads scoping rules and escalation triggers.

Heartbeat: on long tasks the agent checkpoints a write every 30 minutes, so there are no silent completions and every meaningful outcome produces a durable record.

Pattern 3: keeping a long-running fleet’s facts current

caura-long-run-fleet is the 14-day pricing simulation. Beyond the contradiction mechanics, its useful argument is architectural: the agent that writes a fact should not be the agent that validates it. A single agent doing sourcing, verification, and synthesis has no external check on its own output, so a hallucinated fact gets written to memory and recalled as truth every subsequent day. Splitting the roles means a bad write is caught by verification before it ever reaches synthesis, and every write carries an agent_id so the audit trail is per-role rather than per-run.

Engine underneath

caura is the memory layer itself: 12 MCP tools, 14 auto-classified memory types, an eight-status lifecycle (active, pending, confirmed, cancelled, outdated, conflicted, archived, deleted), a knowledge graph with entity resolution that auto-merges above 0.85 cosine similarity, and hybrid recall combining vector similarity, keyword matching, and up to two hops of graph expansion in one call.

It runs standalone with docker compose up and no API key. Connect through MCP and the tools appear in Claude Code, Cursor, Windsurf, or any other MCP client from a single config block.

What breaks in production, and what to do about it

Four failure modes account for most of the pain, and each has a mechanism rather than a workaround.

Duplicate memories crowd out the good ones. Forty agents writing observations about the same customer produces forty near-identical rows, and recall quality drops because the top results are all restatements of one fact. The fix is a crystallizer: an LLM batch process that merges near-duplicates into canonical atomic facts and archives the sources with full provenance. Run it on a schedule you choose.

Contradictions accumulate silently. Covered above. The mechanism is contradiction detection on write, using RDF triple comparison plus semantic analysis, with supersession tracked as a real relationship you can trace rather than a soft signal. In eToro’s deployment the same mechanism runs as a bulk reconciliation pass over the existing pool, retiring conflicted memories and moving others through their lifecycle states without a human adjudicating each one.

PII crosses a boundary it should not. The catch has to happen at write time, before the memory is shareable, not at read time after it has already been indexed. Caura scans on every write and flags detected PII in memory metadata, which is what lets a support-fleet write be quarantined before a sales-fleet recall could ever reach it.

Trust configuration bites you on day one. In caura-build-fleet, the Manager and Code Review agents need trust_level=2 to call the stats, list, and insights tools. Skip the one-time elevation and both agents get 403s and the run reports its isolation check as unconfirmed. This is the correct behavior and it is also the single most common setup failure. Provision agent credentials atomically rather than relying on lazy auto-registration, and set trust explicitly at provision time.

What does orchestration at this scale cost to run?

Three costs, and the second one surprises people.

Tokens: The dominant line item in a naive fleet is repetition, not reasoning. Moving state out of the prompt and recalling only what a task needs is the lever, and the measured effect is a 96.6% to 98.2% reduction against carrying full context.

Latency, multiplied by call volume: A few hundred milliseconds of search latency disappears behind a single LLM call and looks harmless in a demo. Across a fleet issuing recalls before every LLM call, it bills millions of times a day. Caura reports 23 ms p50 and 27 ms p95 warm, which is the number to benchmark any alternative against.

Platform: Caura’s managed tiers are $0 for 10,000 memories with 5,000 writes and 500 recalls a month, $49/month for 250,000 memories, and $399/month for 1 million. Agents and fleets are unlimited on every tier, which matters because per-agent pricing is what makes fleet architectures expensive elsewhere. Self-hosting the Apache 2.0 build costs infrastructure only.

The cost nobody itemizes is the one you are already paying: agents re-deriving what a sibling figured out an hour ago, and engineers manually reconciling contradictions that a lifecycle would have retired.

What does this look like at 300 agents?

eToro (NASDAQ: ETOR) runs the largest production deployment Caura has published. More than 300 specialized agents, 26,500+ memories, and 1,372 shared skills, all against one governed memory layer at 23 ms p50 search.

The operational detail is more instructive than the headline numbers. A QA agent’s test run flagged a hardcoded API key on a staging service and wrote it to memory with org-wide visibility. The security agent surfaced it in its next recall, before a ticket existed and before anyone scheduled a meeting. No handoff was written. No human relayed anything. Fleet health checks run on an automated schedule, and a recurring intelligence brief compiles what the fleet learned that cycle.

Four principles came out of that deployment, and they are the ones worth carrying into your own design. Memory quality beats memory quantity, and the fleet got smarter by curating aggressively. Governance is the enabler rather than the tax, because trust tiers and scopes are what make cross-agent recall safe enough to switch on at all. Skills compound fastest: a workflow written once propagates to hundreds of agents overnight. And the system improves itself, through contradiction detection, lifecycle automation, and per-agent retrieval tuning that are mechanisms rather than maintenance chores.

The full architecture, including the ontology-as-lens idea that lets one event surface as position risk to a trading agent and regulatory exposure to a compliance agent, is in the eToro company brain case study.

Where do you start?

Start with the boundary map, not the framework. Before you pick an orchestrator, write down every agent you plan to run, which department or function it belongs to, and one line naming what it must never be able to recall. That document is your fleet and scope design, and it takes an afternoon.

Then work in this order, because each step makes the next one cheaper:

1. Externalize state. Move shared knowledge out of system prompts into a store agents read and write. Nothing else works until this is true.
2. Stamp scope at write time. Retrofitting visibility onto memories that already exist means reconstructing the intent behind every row by hand. Getting it right on the first write costs one field.
3. Set trust tiers before you add the fifth agent. Defaults are fine for three agents and dangerous at thirty.
4. Write your mandatory rules as policy, not prompt text. Anything a compliance officer signed off on belongs in a deterministic delivery mechanism, not in a paragraph competing with the user’s last message for the model’s attention.
5. Turn on outcome reporting last. It is the compounding step, and it needs the previous four to compound against.

Skip step 5 for a fleet under five agents. The loop needs enough agents for one agent’s lesson to be worth another agent’s while.

Start with the boundary map

You now have the four control patterns, the four ways they degrade past a dozen agents, the mechanisms that fix each one, and four repositories that demonstrate them end to end. The first step is small: list your planned agents, group them into fleets, and write one line per agent naming what it must never recall.

If that list has more than five agents on it, or any row where the answer involves regulated data, the memory layer is the part to get right first. Caura is that layer, Apache 2.0 and free to run against 10,000 memories with unlimited agents. Clone caura-build-fleet if you want to watch constraint propagation work before you commit to anything.

Frequently Asked Questions

What is the difference between AI agent orchestration and workflow automation?

Workflow automation coordinates deterministic steps you defined in advance. Agent orchestration coordinates agents that choose their own next action, which means the orchestrator must handle non-determinism, partial failure, and agents reaching contradictory conclusions from the same inputs. Most production systems combine both: deterministic control flow at the outer layer, agent judgment inside each node.

Do I need shared memory for two or three agents?

No. Per-agent memory with explicit handoffs is fine at that size, and the governance overhead is not worth paying. The pressure shows up somewhere between five and ten agents, when the same fact starts getting re-derived and handoffs start dropping context. Revisit the decision at ten agents rather than at three.

Can agents from different vendors share memory?

Yes, if the memory layer speaks a protocol they all support. Caura exposes its operations over the Model Context Protocol, so any MCP client (Claude Code, Cursor, Windsurf, or a custom runtime) reads and writes the same governed store from one config block. A REST API covers runtimes that do not speak MCP.

How do you stop one agent’s memory from leaking into another’s?

Enforce it at the retrieval layer rather than in the prompt. In Caura, a recall resolves the caller’s authorized fleets into a WHERE fleet_id IN (...) predicate that runs before the hybrid search, so rows outside those fleets are never loaded or scored. Per-row scope_agent ACLs and tenant-level row isolation sit above that for stricter cases.

What happens when two agents write contradicting facts?

The write triggers contradiction detection using RDF triple comparison plus semantic analysis. Whichever memory is older gets marked outdated or superseded, with the relationship tracked so you can trace the chain, and default recall excludes it from results. For a worked example, caura-long-run-fleet shows a pricing change invalidating eight prior memories in one write.

Is this only for large fleets?

The mechanisms pay off with agent count, but the setup cost is low enough to adopt early. Caura’s free tier holds 10,000 memories with unlimited agents and fleets, and the self-hosted Apache 2.0 build runs locally with one Docker command and no API key. Getting scope right on your first hundred memories is cheaper than retrofitting it onto your first hundred thousand.

Related reading: The Token Tax of Multi-Agent Systems · Solving the Agent Cold-Start Problem · How eToro Built a Company Brain