Persistent Memory for AI Agents: From Stateless Prompts to Compounding Knowledge

Persistent memory for AI agents is storage that outlives the session, so an agent begins its next run holding what it learned in the last one. It only works when four things hold at once: the fact can be found, the fact is current, the fact traces back to whoever wrote it, and the fact is visible only to agents allowed to see it. Most implementations ship the first and skip the other three.

This blog covers what persistent memory actually is, the write and recall mechanisms underneath it, the five ways it fails once it is running, why a system prompt cannot govern it, and three open-source repos you can clone today to see each failure and each fix on your own machine.

What is persistent memory for AI agents?

Persistent memory is a store that sits outside the prompt, that any authorized agent can write to and read from, and that survives process restarts, session boundaries, and the agent that created it. The context window is not memory. It is a working set that gets thrown away.

What stateless actually costs

A stateless agent re-learns your environment every morning. It rediscovers that your staging health check lies, that migrations in one region have to run in a specific order, that the billing channel needs a ping before anyone touches it. No model’s training data contains your runbooks or last Tuesday’s incident, so every run starts from the same zero.

The usual workaround is a longer system prompt. That fails on cost before it fails on quality. In a naive multi-agent loop, input tokens scale with agents multiplied by history length multiplied by redundancy, and none of those three multipliers is capability. Attention cost is roughly quadratic in context length, so latency climbs alongside the bill, and models lose the thread in the middle of very long contexts. Five agents with large windows is five times the duplicated spend for noisier answers.

The four properties memory has to hold

Retrieval is the property everyone builds. The other three are where deployments break.

A vector database gives you the first column and a similarity score. It has no opinion on whether the row it just returned was superseded four days ago, which agent wrote it, or whether the agent asking should have been allowed to see it.

What it replaces

Three things, usually.

1. The RAG pipeline over your wiki, because documents go stale and retrieval returns text chunks rather than knowledge.
2. The per-agent scratchpad, because knowledge stays trapped where it was learned.
3. And the shared markdown folder, because a file has no scope, no lifecycle, and no way to detect that two agents wrote contradictory facts into it.

How did agent memory get here, and what did the field miss?

Three papers defined the mental model almost everyone still uses, and all three were written about a single agent.

The founding three

Generative Agents (Park et al., 2023) introduced the memory stream: a complete natural-language record of experience, retrieved by a score that blends recency against importance and relevance, with periodic reflection synthesizing higher-level conclusions. Twenty-five agents populated the sandbox. Each one held its own separate stream. There was no shared substrate between them.

MemGPT (Packer et al., 2023) framed memory as virtual context management, borrowing hierarchical paging from operating systems to move data between a fast in-context tier and a slower external one. The framing is explicit in the abstract: the problem being solved is the limited context window.

CoALA (Sumers, Yao, Narasimhan and Griffiths, TMLR) gave the field its vocabulary: working, episodic, semantic, and procedural memory modules, a structured action space, and a decision procedure. It remains the cleanest map of what memory types an agent needs.

Every one of them is good work. Every one of them models one agent, one user, one long stream.

Gap, counted

We ran a definition audit. We searched the published full text of all three papers (arXiv 2309.02427v3, 2310.08560v2, 2304.03442v2, 67 pages combined) for ten governance terms.

Every non-zero hit is unrelated to memory access. CoALA’s single “permission” is a figure reproduction credit for a Soar architecture diagram. Its four “policy” hits are reinforcement-learning policies. MemGPT’s “policy” is a context-window queue eviction policy. Generative Agents’ “permission” is ACM copyright boilerplate, and its “audit log” appears once in the ethics section, recommending that platforms log inputs and outputs to detect misuse. Not one of the three defines a memory permission primitive.

The field inherited an excellent theory of what an agent should remember and no theory at all of what an agent should be allowed to remember. Teams then deployed fleets on top of it.

Where the shape changed

The three eras are easy to trace: stateless prompt chains needing no memory, persistent single agents needing per-agent memory, then self-improving loops needing structured memory that survives runs. The fourth shape is the one production is now hitting. Dozens or thousands of agents acting for one company, most without a human in the immediate loop, all reading and writing the same state.

At that point memory stops being a retrieval index and starts being a distributed database. Who may read this row, which version is current, where the fact came from, and how it crosses an agent boundary without leaking are consistency questions wearing retrieval clothing. Caura formalized exactly this in a paper on arXiv (2606.24535), which formalizes the fleet-memory problem and names the four failure modes a single-agent memory never has to answer for: unauthorized leakage, stale propagation, contradiction persistence, and provenance collapse.

How does persistent memory actually work?

Three paths: the write, the recall, and the loop that connects them. The specifics below describe Caura’s implementation, which is open source under Apache 2.0, so every claim here is readable in the code.

Write path

An agent sends one field: plain text. One LLM pass then does the structuring before anything lands in storage. It auto-classifies the content into one of 14 memory types (fact, episode, decision, preference, rule, plan, commitment, action, outcome, and others), generates a title, summary and tags, scores importance, scans for PII and flags it, extracts people, organizations, locations and concepts into a live knowledge graph, and compares the new claim against existing rows to detect contradictions. Long content is chunked into atomic facts. The visibility scope is stamped at the door, not applied later.

That last point matters more than it sounds. A scope applied at write time is a column the query planner can filter on. A scope applied at read time is a filter someone has to remember to write.

Recall path

Recall is a pipeline, not a lookup. Semantic candidate generation runs on pgvector, full-text keyword matching runs alongside it, and results expand through knowledge graph relations up to two hops. Ranking combines similarity, importance weight, freshness decay, graph boost and recall boost.

The governance stages sit around that core. The scope predicate runs before search, so rows outside the caller’s authorized fleets are never loaded, never scored, never ranked. Temporal resolution drops superseded rows. Provenance is attached on the way out. The same query returns different rows to different callers by design.

Compounding loop

This is the part that makes month three better than month one, and it is the part most memory layers do not have.

Agents report what happened after acting on a recalled memory. Successes reinforce the memory’s weight. Failures generate a preventive rule that, published at fleet scope, lets the next agents inherit the lesson instead of repeating the mistake. Fleet scope is the intended pattern rather than the automatic default: a generated rule stays private to the agent that wrote it until an agent at trust tier 2 or above publishes it fleet-wide, which is how eToro runs the loop. A crystallizer batch job merges near-duplicate memories into canonical atomic facts, keeping full provenance on the sources it archives. Each agent tunes its own retrieval profile (top_k, min_similarity, graph_max_hops, blend weights) from that feedback, so search quality improves per agent rather than globally.

At eToro (NASDAQ: ETOR), a fleet of 300-plus agents runs on this loop with roughly 291 distinct agent identifiers writing into one memory layer. Current scale: 26,500-plus memories, 1,372 shared skills, 23 ms p50 hybrid search. During one upgrade, the system’s own contradiction detection identified and resolved 3,716 conflicted memories and moved 289 through lifecycle states, without manual curation.

What breaks once memory persists?

Persistence converts every memory bug into a durable one. Five failure modes account for most of what goes wrong, and four of them are invisible in a single-agent benchmark.

Stale propagation

A competitor’s price reads $299 for eight days. Eight memories accumulate, each reinforcing the last. On day nine, the price changes to $349. A raw vector store now holds nine rows, eight of them wrong, and nothing in the similarity score distinguishes current from superseded. The synthesis agent reports $299 with complete confidence. No exception, no flag.

This is a silent failure, which is what makes it expensive. Caura’s write-up on stale memory walks the full 14-day sequence, and the runnable version is caura-long-run-fleet, where three agents (sourcing, verification, synthesis) share one fleet and an async contradiction detector transitions all eight $299 rows to outdated before synthesis queries the pool.

One correction worth carrying into your own implementation: the repo documents that passing status: "active" explicitly is not shorthand for “current,” because the status filter is an exact match and active excludes confirmed rows. The governed query is the default recall with no status filter at all. That is the kind of detail you only learn by running the thing.

Contradiction persistence

Detection working in the lab and detection working in production are different claims, and Caura’s arXiv paper (2606.24535) published the gap rather than hiding it. Both figures below are measurements of the live production service taken for that paper, not results you can reproduce from the OSS build. Contradiction resolution scored a detection rate of 1.000 when both conflicting writes were admitted, and 0.490 across all fact-runs.

The cause was not the detector. A synchronous near-duplicate gate was rejecting the second, contradictory write before the asynchronous contradiction detector ever saw it. An optimization built to suppress noise was suppressing the signal. Publishing that number is the single most useful thing in the paper, because the same pipeline-ordering trap sits in any system that dedups before it resolves.

Scope failure

The same measurement caught a textbook confused deputy. Tenant isolation held everywhere, and the search path enforced the full predicate. The GET-by-id path evaluated only the tenant projection of a row’s scope and skipped the sub-tenant check, so at measurement time a low-trust agent could fetch a cross-fleet row by identifier that the trust ladder should have denied. A handler that resolved the caller’s identity and then failed to use it.

Enforcement was bimodal across API paths. It was disclosed, fixed server-side during the study, and a re-probe found zero cross-fleet reads for the low-trust credential. Every affected number in the paper is flagged “as measured.”

Provenance collapse

When a retrieved fact cannot be traced to its writer, source, and time, debugging becomes guesswork and audits become unverifiable. This one has clean numbers, all measured in the paper (arXiv 2606.24535) against the live production service: across 50 derivation chains at depth four, the measurement rig reconstructed 100% of chains with the correct writer identity at every hop, completeness 1.000 and accuracy 1.000, at 291 ms p50 per hop. Each chain modeled a real incident narrative, from observation to hypothesis to mitigation to verification.

Memory poisoning

Persistence is also an attack surface, and this is the failure mode almost no memory article covers. Unit 42 researchers Royce Lu and Jay Chen published a proof of concept in October 2025 showing indirect prompt injection writing itself into an agent’s long-term memory.

The mechanism is worth understanding precisely. Their test agent summarized each session at the end using an LLM prompt template. The tool output field containing a fetched webpage was the only attacker-controlled input in that template. A payload embedded in the page manipulated the summarization step so malicious instructions landed in the stored summary. Those instructions were then injected into the orchestration prompt of every subsequent session, and the agent silently exfiltrated conversation history to a remote server. Retention in that platform is configurable up to 365 days.

Two design consequences follow. Write-time enrichment is a control point, not just a convenience, because it is where content gets classified and PII gets quarantined before it can cross a boundary. And an audit log answering “which agent wrote this, when, and what happened to it since”, every write, delete, and lifecycle transition, stamped with tenant and scope context, is what turns an incident from unbounded into scoped.

If your agents already write memory from tool output that includes fetched web content, that is the first path to check. Caura enforces PII detection at write time and logs every write, delete, and lifecycle transition with tenant and scope context, which is how the boundary is built rather than bolted on.

Why can’t a system prompt govern memory?

Because retrieval runs before any instruction executes. By the time the model reads “don’t surface legal data,” the legal hold has already been scored, ranked, and placed in context. The instruction is asking a model to un-see something it is currently looking at. Caura’s write-up on why prompt-level data separation is not access control walks the same failure with three agents in one tenant.

Boundary has to be a query predicate

The alternative is enforcement one layer down. Every memory row carries a fleet_id. Every recall turns the caller’s authorized fleets into a WHERE fleet_id IN (...) predicate that executes before hybrid search runs. Rows outside that set are not loaded, not scored, not returned. Model capability has no bearing on whether the boundary holds, which is the entire point.

caura-cross-fleet-gov demonstrates this with three agents on one instance and three partitions. Sales writes a deal to fleet-sales. Legal, scoped to fleet-legal and fleet-org-shared, asks about the same account and gets zero results. Admin, holding all three, performs fan-out recall and merges results with source labels so conflicts are attributed rather than blended.

The repo is also honest about the limit, which is why it is worth reading. In the OSS self-hosted deploy, the boundary holds as long as an agent declares its fleet_ids truthfully; the storage layer filters to what is declared but does not validate the declaration against agent identity. For isolation that cannot be bypassed at the prompt layer, use separate tenants or the managed service, which adds server-side token scoping that rejects out-of-scope recalls at the API layer.

A vendor that documents where its own guarantee stops is telling you something useful about the rest of its documentation.

Three isolation layers, not one

The sensible defaults are one tenant per compliance boundary, one fleet per pipeline or project, and scope_agent reserved for per-agent secrets that must not travel downstream.

Permissions decide who reads; keystones decide what everyone does

Access control answers one half of governance. The other half is the rule every agent must follow regardless of what a user tells it mid-conversation — the case where a customer pushes back five turns in and the agent caves on a policy it was told to hold.

Caura’s answer is keystones: mandatory rules merged across tenant, fleet, and agent scope, fetched deterministically at session start, ordered by weight, and obeyed over conflicting instructions. “Every write carries a retention class” as a tenant keystone governs the whole fleet from one place. Authoring requires an elevated trust tier.

What does persistent memory cost to run?

Two currencies: tokens and latency. Both compound with agent count, which is why numbers that look like microbenchmark trivia at one agent decide viability at a thousand.

The measured numbers

Caura published results on LoCoMo and LongMemEval, the two most-cited public agent-memory benchmarks: 77.6% and 72.5% accuracy under LLM-judge, with 96.6% and 98.2% token savings against full-context baselines, at 23 ms p50 and 27 ms p95 search latency.

The honest framing they put alongside those numbers is more useful than the numbers. Accuracy across the leading memory systems clusters in a narrow band, so for a single chatbot the choice usually comes down to stack fit. Both benchmarks measure one agent, one user, one long conversation.

Neither can ask whether agent 17’s mistake this morning stopped agents 1 through 40 repeating it this afternoon, whether a new agent inherits what the fleet already knows, or whether a sales-fleet memory is correctly invisible to a support agent. Those are the questions that decide whether a memory system is deployable inside a company.

Where the consistency cost lands

Under strong write mode, as measured in the paper (arXiv 2606.24535) rather than in the OSS quickstart, a freshly written fact became visible to authorized readers in effectively one search round trip, 0.83 s p50 and 1.63 s p95. Reads stay fast and the writer pays for correctness, which is the right trade in a fleet that recalls far more often than it writes.

Money

Caura’s pricing puts unlimited agents, fleets and tenants on every tier and charges on storage and calls: Free at 10K memories, 5K writes, 5K searches and 500 recalls per month; Pro at $49/month for 250K memories, 25K writes, 50K searches and 3K recalls; Business at $399/month for 1M memories, 100K writes, 500K searches and 10K recalls. Annual billing brings Pro and Business to $490 and $3,990 a year. Self-hosting the Apache 2.0 engine costs whatever your Postgres and LLM provider bill costs, which for a first evaluation is usually a laptop and nothing.

How do you actually start?

Do not begin by migrating a fleet. Begin by reproducing one failure mode on your own machine, because a failure you have watched happen is worth more than a paragraph describing it.

1. Run one agent against a local instance: git clone https://github.com/caura-ai/caura, docker compose up -d, and write a memory with no API key. The point is to see what a single plain-text write turns into after enrichment: the type, the tags, the extracted entities, the PII flag.

2. Watch a contradiction resolve: Clone caura-long-run-fleet and run the 14-day simulation. Three agents, one fleet, the day-nine price change. Confirm for yourself that the eight stale rows are suppressed by status rather than by prompt instruction, and that the brief reports how many were filtered.

3. Watch a constraint bind a downstream agent: caura-build-fleet runs five agents in sequence: frontend, performance, SEO, code review, manager. The performance agent writes “zero external JavaScript.” The SEO agent recalls it and chooses inline JSON-LD instead of a CDN library. Code review cites both memory IDs in its verdict. Each agent gets an explicit MCP tool allowlist, and the manager never receives the write tool schema at all, so its zero-write audit at the end of the run is a structural proof rather than a promise. The full walkthrough is here.

4. Watch a boundary hold: caura-cross-fleet-gov is the twenty minutes that changes the architecture conversation. Ask the legal agent about the sales account and get zero results, then confirm from the query path that the row was never loaded, never scored, never ranked.

5. Connect a real agent: All of it speaks Model Context Protocol, so Claude Desktop, Claude Code, Cursor or Windsurf take a JSON block with a URL and an API key, and the tools appear immediately. Twelve tools, tenant resolved from the key.

6. Only then decide on scoping: One tenant per compliance boundary, one fleet per project, scope_agent for secrets. Write the first keystone before the fleet grows past five agents, because retrofitting policy across forty agents is a different job.

The whole sequence is an afternoon. It also tells you whether the rest of this applies to your situation, which is worth more than a procurement conversation.

Start with the failure you already have

You now have the four properties memory has to hold, the two paths that implement them, the five ways they break, and the reason a system prompt cannot substitute for a query predicate. The next step is small: pick the failure you have already seen in your own fleet, whether that is an agent acting on a price that changed, two agents shipping contradictory decisions, or a memory surfacing where it should not have, and clone the repo that reproduces it.

Caura is Apache 2.0 and self-hostable, and the managed platform has a free tier with unlimited agents and fleets, so neither path requires a procurement conversation to find out whether governed shared memory changes your numbers.

Frequently Asked Questions

What is the difference between persistent memory and RAG?

RAG retrieves text chunks from documents you already wrote. Persistent memory stores what agents learned as structured knowledge with a type, a scope, a lifecycle status, and a writer. RAG answers “what does our documentation say”; memory answers “what did we find out, who found it, and is it still true?” Most production systems run both.

Is a vector database enough for agent memory?

For one agent, often yes. For a fleet, no. A vector store has no concept of who should see what, no contradiction detection, no lifecycle, and no provenance. It returns a similarity score, which cannot tell you whether the row it just surfaced was superseded last week or written by an agent that should not have had access to the subject.

Can I just use a bigger context window instead?

A bigger window raises the ceiling on a bill that should not exist. It does not give you scoped access, temporal correctness, provenance, or safe propagation between agents, and cost plus latency still climb with every token placed in it. The token math is here.

How do agents share memory without leaking data?

Through scope stamped at write time and enforced as a query predicate at recall time, plus trust tiers per agent and an audit log covering every write, delete, and lifecycle transition. The important property is that enforcement happens in the database query, before retrieval scoring, rather than in a system prompt that runs after retrieval has already loaded the row into context.

Can persistent memory be attacked?

Yes. Indirect prompt injection can write malicious instructions into stored memory that then persist across sessions, which Unit 42 demonstrated in October 2025 against a session-summarization pipeline. Treat any tool output that reaches a write as untrusted, enforce classification and PII detection at write time, and keep an audit trail so an incident has a boundary.

How long before shared memory pays off?

The write path pays immediately, because one enrichment call replaces the classification, tagging and entity extraction you would otherwise build. The compounding pays later, once outcome feedback and crystallization have run over real traffic. eToro’s cleanup pass resolving 3,716 conflicted memories is the shape of that second phase, not the first week.

Related reading: AI Memory Is a Distributed-Systems Problem · What Is Agent Fleet Memory? · Beyond System Prompts: Keystones