Multi-Agent Memory: How Shared Context Works Without Agents Talking to Each Other

Multi-agent memory works by turning every handoff into a record instead of a message. One agent writes what it learned into a shared, governed store. Another agent recalls it when that agent runs, minutes or weeks later, having never exchanged a word with the writer. Coordination happens through the substrate, not across a wire.

The published run of caura-ai/caura-build-fleet shows the shape: five agents complete a pipeline in 147.5 seconds using 19 memory operations and zero messages between agents. The SEO agent honors a “no external JavaScript” rule the Performance agent wrote, and the two never met.

This blog covers the part most people don’t talk about: the contract a record has to satisfy for a stranger agent to act on it, how long a write takes to become safely readable, and the four cases where the substrate is the wrong tool and you still need a channel.

What does “without talking to each other” actually mean?

It doesn’t mean the agents are unwired. Something still starts them, sequences them, and collects their output. That is the orchestrator’s job, and orchestrators pass messages constantly.

The distinction is about knowledge, not control flow.

In a message-passing handoff, Agent A must know that Agent B exists, must decide B needs this fact, and must say so before A’s session ends. Three chances to fail. In a substrate handoff, A writes the fact to a store and stops. B queries the store when B needs to know something. A never learns B exists.

This pattern predates LLMs. It is the blackboard architecture, where specialists post to and read from a shared workspace rather than addressing each other. A multi-agent blackboard paper from Google and UMass Amherst researchers rebuilt it for LLM agents: a central agent posts a request to a shared blackboard and subordinate agents volunteer based on their own capabilities. The measured gain was 13% to 57% relative improvement in end-to-end success on data-discovery benchmarks.

The authors are explicit about why it works, and the reason is organizational rather than statistical: the design “remov[es] the need for a central coordinator to know each agent’s expertise or internal knowledge.”

Anthropic reached the same conclusion from production. In the appendix on how they built Claude’s Research system, in a bolded point titled “Subagent output to a filesystem to minimize the ‘game of telephone’,” they recommend that subagents write their work to an external artifact system and pass only lightweight references back to the coordinator. Their stated reason: routing everything through the lead agent loses information and burns tokens copying large outputs through conversation history.

Two independent teams, one academic and one shipping to millions of users, arrived at the same move. Take the payload off the message bus and put it in a store.

Why does message passing break before the memory layer does?

Three reasons, and they compound.

The link count grows quadratically. Eight agents that coordinate by messaging have 28 possible pairs to keep correct. Sixteen have 120. Every one is a place where a fact can fail to travel. With a shared substrate, eight agents have eight connections, and adding a ninth adds one.

Withheld information is a top-tier failure mode. Cemri et al. built the first empirically grounded taxonomy of multi-agent failures from 1,642 annotated traces across seven frameworks, validated against expert human annotators at a Cohen’s kappa of 0.88. They identify 14 failure modes in three categories, one of which is inter-agent misalignment: failures from ineffective communication, poor collaboration, and conflicting behavior between agents.

Failure mode 2.4 is Information Withholding, and the paper’s own finding about it is the important part. It “appear[s] almost exclusively in failed runs.” In their ChatDev data, it shows up in 0.0% of successful runs and 5.0% of failed ones; for MetaGPT, 0.0% and 5.6%. Their illustrative trace is mundane: a Phone Agent fails to tell a Supervisor Agent about a username format requirement, the Supervisor fails to ask, and the task dies in a loop of failed logins.

A substrate does not fix an agent that fails to record something. It does remove the second half of that trace. There is no recipient who has to think to ask, because the query is part of the reader’s normal task loop.

Messages are expensive. Anthropic reports that multi-agent systems use roughly 15 times the tokens of a chat interaction, and that token usage alone explains 80% of performance variance on the BrowseComp evaluation. When coordination payload rides in conversation history, every relay pays for the same content again. This is the token tax fleets discover in month two.

A position paper from UCSD and Georgia Tech, framing multi-agent memory as a computer architecture problem, puts the constraint succinctly: “inter-agent bandwidth remains limited by message passing.” Their conclusion is that the connectivity layer, including MCP, is necessary but not sufficient, and that the field lacks a memory access protocol.

What has to be in the record for a stranger agent to act on it?

The UCSD paper names the gap precisely, as three unanswered questions: can one agent read another’s long-term memory, is that access read-only or read-write, and what is the unit of access? Those are not abstract. They are the fields a record needs, and an implementation that ships has to answer all three.

A record is a message with the recipient removed. Everything the recipient would have supplied from context now has to be in the row.

Four fields do the work a conversation used to do implicitly.

Scope replaces the addressee. The reader is a permission set rather than a name, which is what lets a record reach an agent provisioned six weeks after the write. In Caura this is stamped at write time as scope_agent, scope_team, or scope_org, and the cross-fleet governance demo shows what that buys: three agents on one backend where the Legal agent’s compliance holds return zero results for the Sales agent, because the fleet predicate runs before the search, not after.

Status and weight replace tone. A human hedges in prose and the reader calibrates. An agent cannot parse a hedge reliably, so the record itself has to say how far to rely on it. In Caura, that is a lifecycle status such as pending, confirmed, or outdated, plus a numeric weight the recall ranker uses. Trust is a separate control that lives on the agent: a 0 to 3 permission level deciding what that agent may read and write.

Provenance replaces the sender. Which agent, on what evidence, and when. Without it you cannot revoke a bad belief, and one confident hallucination becomes the fleet’s official position with nothing to trace it back to.

Validity replaces the timestamp. Two dates, not one: when the fact was true in the world and when the system learned it. Collapse them and “our price was $40” and “our price is $55” read as a contradiction instead of a history.

Strip these, and you have a sentence in a vector index, which is exactly the failure mode covered in why a vector database is not enough. The full taxonomy of what carries scope, provenance, trust, and validity is worth reading alongside this.

How long after a write is the fact actually shared?

This is the question nobody publishes, and it has a real answer.

An indirect handoff is asynchronous by construction, so there is a settling window between the write returning and the fact being safely readable. Reading the published latency tables and API behavior in caura-ai/caura and the polling logic described in caura-long-run-fleet, the window has four stages.

The commit is synchronous and returns an ID. Everything after it is not. LLM enrichment, which classifies the memory type and generates the title, tags, PII flags, and extracted entities from a single content field, dominates write latency at roughly 2,000 ms p50. Embedding runs asynchronously behind a metadata.embedding_pending flag, so a just-written memory may not surface in semantic search for a moment after the write returns.

Contradiction detection is also asynchronous, and it is the one that matters most: until it completes, the store holds the old fact and the new one side by side, and recall will return both.

Two operating rules follow from this, and both are cheap.

If the next agent runs immediately, list by metadata rather than by semantic search. The non-semantic list surfaces the row the instant it commits, before the vector exists. Semantic recall is the right default everywhere else; it is the wrong default inside the settling window.

If the write contradicts a held belief, gate the reader on the contradiction pass. The long-run-fleet simulation does exactly this. Its simulate.py polls GET /memories/{memory_id}/contradictions until detection_status returns completed before the Synthesis agent is allowed to query. Skip that, and Synthesis reasons over a pool holding both prices.

Treating the store as instantaneous is the most common mistake in a first fleet. It is a distributed system, and it has a visibility model regardless of whether you have looked at it.

What does a zero-message handoff look like in a real run?

Two of Caura’s reference implementations show it end to end, and both publish their output.

Constraint propagation, in one pass: The build-fleet pipeline runs Frontend, Performance, SEO, Code Review, and a read-only Manager. Frontend writes three structural decisions and recalls nothing, because nothing precedes it. Performance recalls those decisions and writes the rule that there will be no external JavaScript. SEO recalls the fleet and, seeing that rule, chooses inline JSON-LD instead of a CDN schema library. Code Review recalls everything, runs contradiction detection, and issues a verdict citing the specific memory IDs it rests on.

Nobody told SEO about the JavaScript ban. SEO went looking, and it was there. That single substitution is the whole pattern.

The Manager agent is the part worth copying. It has no write tool in its schema at all, so read-only isolation is enforced by tool availability rather than by instruction. At the end of every run, it reports zero write operations, which is a proof rather than a promise.

Supersession, over fourteen days: The long-run-fleet simulation runs Sourcing, Verification, and Synthesis daily. A competitor’s price reads $299 for eight days, then changes to $349 on day nine. Sourcing writes the new number. Contradiction detection marks all eight prior memories outdated. On day ten, Synthesis recalls and gets one result. The eight stale records are suppressed by status, not by a prompt instruction telling the model to ignore old data.

That difference decides whether a fleet survives a quarter. A prompt asking an agent to disregard stale facts does not remove stale vectors from the result set. A status transition does.

In production, the numbers get larger without changing shape. eToro runs 300+ agents on one governed memory with 26,500+ memories and 1,372 shared skills.

Where does indirect coordination fail?

Four cases, and pretending otherwise wastes a month.

Mutual exclusion: Two agents editing the same file need a lock, not a memory. A record saying “I am working on this” is advisory and arrives late.

Strict ordering: If B must not start until A finishes, that is the orchestrator’s job. Checkpoints and graph edges exist for this, and they are good at it.

Sub-second interlock: When the settling window is longer than the task, the substrate cannot carry the handoff. Measure before you assume.

Negotiation: Bidding, debate, task assignment. These need turn-taking, and turn-taking needs a channel.

There is a fifth case worth highlighting because the code is honest about it. The cross-fleet governance repo publishes a deliberate “what if the agent lies?” test.

In the open-source deployment, the storage layer filters recall to whatever fleet_ids the caller declares, and doesn’t validate that declaration against the agent’s identity. A Sales agent that declares fleet-legal gets legal records back. The boundary holds because each agent’s AGENTS.md forbids declaring fleets it doesn’t own, which makes it a query-layer contract rather than a cryptographic one.

The design lesson generalizes. Indirect coordination moves enforcement from the prompt to the query layer, but only if the substrate validates the reader’s identity on the server side. Otherwise you have relocated the trust assumption rather than removed it.

Hard isolation needs separate tenants, per-row ACLs, or agent-scoped credentials minted with their scope bound at issue time, which is what Caura’s managed tier and the POST /admin/agent-keys/provision path do. The same principle drives keystones, which are injected at session start rather than retrieved, so a policy cannot lose a ranking contest against a marginally relevant memory.

If enforcement lives in a system prompt, it is advice. Queries are not advice.

How do you actually set this up?

Five steps, in an order that avoids the common retrofits.

1. One store, not one per agent. Even a single shared table beats N private ones. Retrofitting isolation later is brutal.
2. Scope, tenant, and provenance on every record from day one. These three take an afternoon and prevent most of the pain.
3. Make recall the first step of every agent’s task loop. Knowledge nobody retrieves is knowledge nobody has. This is the step teams skip, and it is the step that makes the sharing real.
4. Decide your settling policy. Which handoffs can tolerate the async window, and which need a metadata list or a contradiction-detection gate.
5. Add trust tiers, supersession and a dozen keystones once the fleet is writing daily. Not before, because you will not know what to write.

Caura is Apache 2.0 and speaks MCP, so any runtime that speaks MCP gets the same twelve memory tools with no per-framework adapter. A local Docker deployment writes and recalls its first memory in about four commands, with no API key. Start free or run the engine yourself.

Conclusion

You now have the contract a record needs to replace a message, the settling window between a write and a safe read, and the four cases where you should not try. Start with step three above: put a recall call at the top of one agent’s task loop and see what it finds. If it comes back empty, your fleet has been paying for the same discoveries repeatedly and had no way to notice.

If you would rather not build the right gates, the supersession logic, and the audit trail yourself, that is what Caura is. Apache 2.0, self-hosted or managed, and the free tier holds 10,000 memories across unlimited agents and fleets.

Frequently Asked Questions

Is multi-agent memory the same as RAG?

No. RAG retrieves from a corpus that humans or pipelines wrote. Multi-agent memory is written by the agents themselves as they work, which means it needs things a corpus does not: provenance for revocation, status and weight for deciding how much to rely on a record, supersession for facts that change, and scope for who may read what. Both use vector search underneath. The governance on top is the difference.

Do agents need to know which other agents exist?

No, and that is the point. An agent writes to a scope and reads from a scope. It never addresses a peer. This is why a subagent spawned five minutes ago has the same recall as a long-running lead, and why adding the ninth agent to a fleet costs one connection instead of eight.

How is this different from a shared file or scratchpad?

A scratchpad persists and is readable, which covers two of the four requirements. It has no supersession, so stale notes sit next to current ones with equal authority, and no provenance, so you cannot tell who wrote what or revoke it. Caura’s post on the five ways agents share knowledge compares them across those axes.

Does this replace my orchestrator?

No. Let the orchestrator own coordination and let the memory layer own knowledge. Checkpoints stay for resumability; durable findings go to the store. Keeping memory below the orchestrator also means switching frameworks does not reset what the fleet knows.

What is the smallest useful version of this?

One shared store reached over MCP, with a scope tag on every record, and a recall call at the top of each agent’s task loop. Nothing else is required on day one. Add the governance once writes are happening daily, rather than designing an ontology first.

Related reading: How AI Agents Share Knowledge · What Is Agent Fleet Memory? · Why a Vector Database Is Not Enough