What Is an AI Memory Layer?

Most systems described as having memory are replaying a transcript. That works until the conversation outgrows the window, at which point the earliest and often most important turns are the first to be dropped. A memory layer is the thing that stops decisions evaporating when the buffer fills.

Three kinds of state, commonly confused

• Working state — the current conversation. Ephemeral, bounded by the window, safe to lose when the task ends.
• Episodic memory — what happened: a summary of an interaction, retrievable later. Useful, and the easiest to let grow into noise.
• Semantic memory — durable facts and decisions: this customer is on the enterprise plan, we chose Postgres over DynamoDB and here is why. This is the layer that changes behaviour.

The test for what belongs in memory

Ask whether the fact should still influence behaviour in six months. "The user said hello" fails. "The user's deployment target is EU-only for data-residency reasons" passes, and if it is not durable, you will violate it the first time the transcript is trimmed.

What a memory record needs

• A claim — one fact or decision, stated plainly.
• Provenance — who or what established it, and when. Without this you cannot resolve contradictions.
• Scope — who or what it applies to: this user, this project, this organisation.
• A status — current, or superseded by a later record. Never edit in place.
• A trigger for review — the condition under which it should be reconsidered.

The contradiction problem

Memory that only appends eventually holds two incompatible claims, and retrieval will happily return both. Some systems resolve this at read time by preferring the most recent; that is wrong as often as it is right, because recency is not authority. Superseding explicitly — marking the old record and pointing at the new one — keeps the history readable and the current state unambiguous.

Retrieval, not injection

A memory layer that dumps everything it knows into every prompt has recreated the context-dilution problem it was meant to solve. Memories are retrieved like any other context: scoped, ranked, and limited. If a memory is not relevant to the current task, its correct contribution is nothing.

Where this overlaps with documentation

A durable record of a decision, its context and its consequences is an architecture decision record. The difference is the reader: an ADR is written for humans and happens to be machine-readable, while a memory record is written for retrieval and happens to be human-readable. Teams that already write ADRs have most of a semantic memory layer and usually have not indexed it.