Long-Term Memory for AI Agents: Which Architectures Survive Contact With Production

Long-term memory for an AI agent is decided at the write path and the lifecycle, not at retrieval, which is where almost every published architecture puts its effort. Anthropic’s own figures make the point cleanly: in a 100-turn evaluation, clearing stale context alone improved performance by 29%, while adding a persistent memory tool on top of that brought only another 10 points, to 39%. Pruning did three-quarters of the work.

That ratio is the thesis of this blog. A long-term memory AI system that accumulates is easy to build and quietly degrades. One that admits carefully and retires facts on purpose is harder and holds up. What follows is every architecture family worth knowing, what each one persists, the measured failure modes of long-term memory in production, and how to choose between them.

What counts as long-term memory for an AI agent?

Long-term memory is a store outside the model that survives the call, the session and the process, plus the machinery that writes to it and takes things back out. The model itself contributes nothing: an LLM holds no state between requests, so everything that feels like memory is something the surrounding system put back in.

Four layers, and only two of them persist

The vocabulary borrowed from cognitive science transfers unevenly, so it is worth being precise about what each layer is in an engineering sense.

• Context window. Scratch space for one call. Wiped when the call ends.
• Working memory. The current run’s state: file paths, variable names, the tool output two steps back. Lossless and exact, discarded when the task finishes.
• Episodic memory. What happened: this agent attempted that migration on Tuesday and it failed this way.
• Semantic memory. What is true: the rate limit is 300 requests a minute, the customer is on the enterprise plan.

Only the last two are long-term. Caura’s explainer on why the context window is a working set, not memory, draws the same line from the storage side. The practical consequence is that episodic and semantic memory have opposite requirements, and systems that use one store for both get worse at both. Episodic entries are append-only and never become false; they only become irrelevant. Semantic facts are the opposite: there is one current answer, and yesterday’s is actively harmful.

A bigger context window is not long-term memory

This is settled empirically, and the numbers are stronger than the intuition. In the TACL paper Lost in the Middle, Liu et al. measured GPT-3.5-Turbo on multi-document QA with 20 documents. When the answer sat in position one, accuracy was 75.8%. In the middle, at position ten, it was 53.8%. Closed-book accuracy, with no documents supplied at all, was 56.1%.

Read those three numbers together. Retrieving the correct document and placing it mid-context performed worse than retrieving nothing. More context is not more memory, and in the wrong position it is negative.

LongMemEval (Wu et al., ICLR 2025) isolates the same effect for agents. Its 500 questions come with two haystack settings, LongMemEval_S at roughly 115k tokens and LongMemEval_M at around 1.5M. The paper compares each model against an oracle condition where only the evidence sessions are supplied. GPT-4o scored 0.870 on oracle and 0.606 on the full history. Llama 3.1 70B went from 0.744 to 0.334. The evidence was present in the context in both conditions. The only variable was how much surrounding history came with it.

Which long-term memory architectures exist today?

Seven long-term memory families, each with a published mechanism. Grouping them by what they persist is more useful than grouping them by storage technology, because the storage choice follows from the lifecycle requirement rather than the other way round. All seven solve retrieval, and they solve it well. The column that separates them is the last one: what happens when a stored fact stops being true.

Virtual paging, memory streams and graph retrieval have no answer at all. Engineered decay has an answer based on age, which handles abandonment but not contradiction, because a fact that is recalled constantly and is also wrong gets reinforced rather than retired. That leaves the governed-store family, where supersession is a first-class operation. Caura’s write-up on persistent memory and how it breaks covers the same ground from the failure side.

Why is the write path harder than retrieval?

Because what reaches the memory store determines everything downstream, and agents are bad at deciding what to keep. It is the least-covered part of the topic and the one where a design decision changes the most.

Agents left to record their own work keep almost nothing useful

Caura’s read-only analysis of an eToro deployment snapshot dated 25 August 2026 found that 96% of what an agent logs for itself is disposable episodic telemetry, recalled essentially never. The reason is structural rather than a prompting failure: journaling competes with the task, and under task pressure the agent does the task. Caura’s piece on why agents don’t journal makes the case that any memory write left to the model’s discretion is a memory write that will be skipped when it is most needed.

The same snapshot shows how much of what the fleet holds came from a scheduled reflection pass rather than from an agent’s own write. Of every decision-grade memory in the system, 66% of preferences, 48% of decisions, 46% of tasks, 38% of outcomes and 29% of facts were recovered from work no agent hand-wrote. The ordering is informative. Preferences are the type agents are worst at recording deliberately and the type that most changes future behaviour.

Extraction corrupts data before storage ever sees it

The strongest practitioner evidence here is not from a vendor. On Hacker News, an agent developer posting as endymi0n described abandoning fact extraction after the extracted database became, in his words, a mess of incomplete and invalid sentences. His examples are specific. Irony and sarcasm extract as sincere facts, dates come out wrong, and a stored fact like “flights are booked for the Paris trip” survives unchanged after the trip is rerouted.

The reply in that thread is sharper still, because it comes from the author of the memory database being discussed, conceding the point against his own product: if the agent is producing brittle inferred facts upstream, no memory backend saves it. A store can manage rot and contradiction. It cannot fix bad inference.

That is the architectural rule. An admission policy that decides what is worth keeping, applied in the runtime rather than left to the model, outranks every retrieval choice you will make. Caura argues the same in deterministic versus agentic memory: a memory write that depends on model behaviour is a reliability bug.

What practitioners complain about most often

We searched every Hacker News comment mentioning memory in an LLM or agent context between October 2025 and October 2026, filtered to comments over 200 characters that pair memory with an agent or LLM term and exclude hardware discussion. That leaves 929 distinct comments.

The most common complaint is not retrieval quality. It is that nobody can explain why a given memory surfaced, which appears in 34 comments. Next is the cost of replaying history (33), then stale long-term memory being served as current (31), then quality degrading as the store grows (28), then contradictory facts accumulating (22). These are keyword-matched counts over a filtered corpus, so treat them as indicative rather than formal content analysis. The ordering still stands up: provenance and freshness beat recall accuracy in what engineers report going wrong.

How does a memory layer know a fact has gone stale?

Through contradiction detection and supersession, and this is the stage where published long-term memory AI architectures are thinnest. We reviewed the 17 commercial and editorial pages ranking across three long-term memory queries on Google US in October 2026, excluding forums, videos, and papers. Not one of them mentions TTL by name. Only one describes supersession as a mechanism rather than a sentence. Seven cover no lifecycle at all.

The paraphrase blind spot, with measurements

Contradiction detectors pull candidate memories above a cosine similarity floor and hand them to a judge. The floor is almost always set by intuition, and intuition sets it too high.

Caura’s engine carries a source comment in common/constants.py recording what happened when it measured, and a test that asserts the same four numbers. Four genuine, deliberately paraphrased updates were written through the real pipeline and scored against the facts they superseded: 0.483, 0.503, 0.568 and 0.607. Every one fell below the 0.70 floor then in use, so the candidate query returned zero rows and the judge never saw the contradiction it exists to catch.

The threshold is now 0.45, and the fix costs nothing, because the candidate query is bounded by its LIMIT rather than by the threshold. Lowering the floor cannot add a twenty-first candidate. It changes which rows fill the window.

If you take one implementation detail from this article, take that one. An update phrased differently from the fact it replaces is exactly the case a similarity floor is meant to catch, and it is exactly the case a floor set by intuition misses.

Decay is not one clock for everything

The second lifecycle mistake is a single global recency curve. Caura sets freshness decay per memory type: a stated preference decays over 365 days, a cancellation over 14. That is a 26-fold spread between two types sitting in the same store. Apply one curve to preferences, prices, cancellations, and decisions, and you are either forgetting preferences too fast or serving cancelled commitments as current.

Timestamps need the same care. Caura’s as-of recall exists because importing two years of tickets in one afternoon gives every memory the same creation date, so freshness becomes a constant across the whole import and a query like “last month” resolves to a window ending on the import day. Separating event time from ingest time is the fix, and it becomes load-bearing the first time you backfill history, which is to say the first time a real customer onboards.

Correct components in the wrong order still fail

Caura published a measurement of its own production service as an arXiv paper, Governed Shared Memory for Multi-Agent LLM Systems, written up under the project’s former brand name, and the most useful result in it is a bug. Contradiction detection scored 1.000 when both conflicting writes were admitted, but 0.490 across all fact runs. The ceiling was set by write admission: a synchronous near-duplicate gate returned 409 on 206 of the 400 writes before the post-commit contradiction detector saw them, so only 98 fact runs had both writes land. Two individually correct components, wrong order, and an optimisation meant to suppress noise was suppressing the signal. The paper notes it cannot fully separate that effect from limited prose diversity in its own workload generator.

That failure is invisible to every retrieval benchmark, because the contradictory fact never entered the store to be retrieved. It is the kind of thing only live measurement finds, which is an argument for instrumenting your memory layer rather than trusting a leaderboard.

What do the long-term memory benchmarks really tell you?

Less than the numbers suggest, and cross-vendor comparisons are not comparable at all. Four of the 17 pages we reviewed publish any benchmark number, and every one publishes its own system’s score against its own pipeline.

The scores move with the setup, not just the memory

Different long-term memory systems use different ingestion, different readers, different judges and different retrieval budgets. An April 2026 independent audit by Penfield Labs makes the point concretely: across LoCoMo’s answer key, 99 of 1,540 questions carry score-corrupting errors, 6.4% of the set, which puts the theoretical maximum for a perfect system at roughly 93.6%. The same audit generated intentionally wrong but topically adjacent answers and found the standard GPT-4o-mini judge configuration accepted 62.81% of them. Vague answers that named the right topic and missed every detail passed nearly two thirds of the time.

Note what that does to the field. Drift toward vagueness is the failure practitioners report most, and it is the one the standard evaluation is blind to.

Needle-in-a-haystack is not a memory benchmark

HELMET (Princeton and Intel, ICLR 2025) evaluated 59 long-context models across seven application-centric task categories and found that synthetic tasks like needle-in-a-haystack do not reliably predict downstream performance. Most models score perfectly on the needle and still fail realistic long-context work. If a vendor’s evidence for its long-term memory is a needle test, it has not measured the thing you care about.

What a credible number looks like

Caura publishes its LongMemEval result at 92.2%, 461 of 500 under a GPT-4o judge and 451 of 500 under a second judge, with the evaluation code, the committed result files and the judge-selection protocol all public. It also runs an oracle condition, where perfect retrieval is handed to the same reader, which scores 94.6%. Retrieval costs 2.4 points and the reader costs the remaining 5.4.

The full set of approved claims, with caveats attached, sits in its benchmark evidence registry, which also records which of its own previously published memory numbers have been withdrawn and why. Whatever long-term memory system you evaluate, that is the shape to ask for: the judge, the reader, the protocol, and an oracle condition showing how much headroom retrieval even has.

What changes when several agents share one long-term memory?

The failure modes change category. Single-agent long-term memory is a storage problem. Shared memory is a distributed-systems problem, and only 4 of the 17 pages we reviewed cover it at all, three of those in a single sentence.

Four failures a single-agent store never has to answer for

The governed shared memory paper names them: unauthorised leakage, stale propagation, contradiction persistence and provenance collapse. Each needs a primitive rather than a policy document. Scoped retrieval answers leakage, temporal supersession answers staleness, provenance tracking answers collapse, and policy-governed propagation answers the rest.

Provenance is the one practitioners ask for unprompted. In the paper’s measurement, all 50 four-hop derivation chains were reconstructed with the correct writer identity at every hop, completeness and accuracy both 1.000, at 291ms p50 per hop. On a separate Hacker News thread about a shared memory engine, a commenter asked for exactly this without being prompted: which agent created a memory, which agents later retrieved it, and how it propagated, specifically to track down stale or conflicting entries.

A declared scope is not an enforced one

Caura’s cross-fleet governance reference is unusually honest about its own limit, and the same limit applies to most shared memory designs. Fleet isolation works because the fleet identifier becomes a filter applied before the search runs, so excluded memories are never loaded, never scored, and never returned.

In the self-hosted open-source deploy, though, the storage layer filters to whatever the agent declares, and does not validate that declaration against the agent’s identity. It is a query-layer contract, not a key. Structural invisibility needs a tenant boundary or a trust tier, which is why Caura’s four trust levels sit below the scope system rather than beside it.

The same paper found a second, subtler version in production: scope enforcement was bimodal across API paths. The search path carried the fleet predicate but enforced it only partially, with 43.9% of cross-fleet probes still surfacing the row, while a fetch-by-id checked only the tenant projection and discarded the caller’s identity. A classic confused deputy, surfaced by a formal probe rather than by use, and remediated server-side the day after it was measured.

How do you choose a long-term memory architecture for your system?

Five questions, in this order. Answering them out of order is how teams end up with a long-term memory store nobody trusts, and nobody can remove.

1. Is this episodic or semantic? Episodic memory is append-only and can grow; semantic memory has one current value and needs supersession. Mixing them in one store with one policy degrades both.
2. Who decides what gets written? If the answer is the model, expect to keep almost nothing useful. Move the write into the runtime, or add a scheduled reflection pass over the work trail your agent runtime already keeps.
3. What happens when a stored fact becomes false? If the answer is a delete or an overwrite, you have lost the audit trail. If there is no answer, the store will drift. Supersession keeps both the current answer and its lineage.
4. How fast does each type of fact go out of date? Set decay per type. One curve for everything is wrong for almost everything.
5. Does more than one agent read this? If yes, scopes, write attribution and provenance stop being nice to have, and they are much harder to retrofit than to design in.

If you want to see those five running rather than read about them, Caura’s engine is Apache 2.0 at github.com/caura-ai/caura, the benchmark code is reproducible, and the measurement tooling behind the paper is public too. Its two-week fleet pilot is scoped to one workflow and five to twenty agents, which is about the smallest setup where governed long-term memory earns its cost.

Conclusion

You can now tell the difference between a long-term memory AI system that will hold up and one that will merely accumulate. Every architecture family above solves retrieval. Those that persist are the ones that decide carefully what to admit, type what they store, and take a fact back out when it stops being true.

The first step is small enough for a couple of hours. Take one agent that repeats work, pull a week of its logs, and sort what it recorded into facts that are still true, facts that have since changed, and telemetry nobody will ever read. If the third pile is most of it, your problem is admission. If the second pile is serving as current, your problem is lifecycle. Either way, you now know which half to build, and Caura exists because at fleet scale you eventually need both.

Frequently Asked Questions

What is long-term memory in AI agents?

Long-term memory is a store outside the model holding facts, decisions, preferences, and outcomes that survive the session, together with the write path that fills it and the lifecycle that retires entries. The model contributes nothing, since an LLM holds no state between requests. Everything that feels like memory was put back into the prompt by the surrounding system.

Is a vector database enough for long-term memory?

No, and treating the two as equivalent is the most common mistake on this topic. A vector database is a read path. It has no opinion about what should have been written, no mechanism for noticing that a stored fact has been contradicted, and no way to retire one. Those are separate systems you have to build or buy alongside it.

Does a larger context window remove the need for a memory layer?

No. On LongMemEval, GPT-4o scored 0.870 when given only the evidence sessions and 0.606 when given the same evidence inside the full history. The evidence was present both times. Larger windows raise the ceiling on a single call and do nothing about what survives it.

What is the difference between episodic and semantic memory for agents?

Episodic memory records what happened, such as an agent attempting a migration on a given day. Semantic memory records what is true, such as a current rate limit. Episodic entries never become false, only irrelevant. Semantic facts have exactly one current value, which is why they need contradiction detection and supersession while episodic entries do not.

How do you stop an AI agent’s memory from going stale?

Detect contradictions on write rather than at read time, supersede the old fact instead of deleting it, and set freshness decay per memory type rather than globally. The detail that catches people out is the similarity floor on contradiction candidates, which not one of the 17 ranking pages we reviewed discusses: measured paraphrased updates scored between 0.483 and 0.607 against the facts they replaced, well below the 0.70 floor many systems default to.

Are LoCoMo and LongMemEval scores comparable across vendors?

No. Each system uses its own ingestion, its own reader, its own judge and its own retrieval budget, so the evaluation setup moves the number as much as the memory does. An independent audit also found 6.4% of LoCoMo’s answer key is wrong, capping a perfect system at roughly 93.6%, and that a standard judge accepted 62.81% of intentionally wrong answers. Ask for the protocol, not the headline.

Related reading: Persistent Memory for AI Agents · Caura on LongMemEval · Stateful vs Stateless AI Agents