Memory for AI Agents is a filing system, not a file

As CTO of firsthand, my job was to hold a credible viewpoint on an impossibly wide range of topics. In a single week I might need to track state level healthcare-policy changes, evaluate new smart locks for the offices, keep pace with an AI dev tool that shipped on Tuesday and was stale by Friday, and decide whether to buy or build some new RCM tool. The surface area is absurd.

So I tried to build an assistant to carry the context I couldn't. Remember what I'd already researched, what we'd already decided, what had already failed, so I wouldn't restart from zero every time a topic resurfaced. It disappointed me in the same specific way every time, and it took me a while to name the problem: the thing had unreliable memory. I couldn't load all of this into context or build an MCP for this wide range of topics.

Every time an AI agent starts a session, it wakes up with amnesia. The naive fix is "just put everything in personal CLAUDE.md." That works until the file is 4,000 lines long, half of it is stale, and the agent confidently follows a rule that stopped being true three months ago. The Claude and OpenAI apps gradually add information about you to context but it is opaque to the user and that - context isn't free, and wrong context is worse than no context. The apps leave you no way to fix incorrect context that accumulated.

The assistant taught me that the hard part of a useful AI isn't intelligence, it's building an accurate fact base. The place I finally saw it solved with real engineering discipline is Kelp (an AI content-aggregation platform), which has grown a layered memory system to deal with exactly this. None of the individual ideas are exotic. However, they are deployed for specific problems and treated as a real subsystem, with the same discipline we'd apply to a database: tiered access, write protocols, and most importantly garbage collection.

What makes Kelp worth writing up: there are actually two completely different memory systems living in the same repo, built by different "minds" for different reasons.

1. The builder's memory: how the AI coding agent (and human contributors) retain knowledge about the project across sessions. This is documentation, rules, and decision logs: the meta-layer used to build Kelp.
2. The application's memory: how Kelp-the-product, which crawls the web and summarizes content, retains synthesized knowledge about the content it has already read, so it doesn't re-read and re-reason from scratch on every run.

They sound unrelated but are really doing the same thing: version don't mutate, cite your sources, know when you're stale, and budget what you load. Good memory engineering looks the same whether the thing remembering is an agent or an app.

This post walks through both, with real examples from the repo.

---

Part 1 — The builder's memory

The core idea: memory is a subsystem, not a file

The agent guidelines file (AGENTS.md) opens with a line that sets the whole tone:

I'm an expert software engineer whose memory resets between sessions. I rely entirely on the Memory Bank to understand the project. Reading these files at the start of every task is mandatory.

A competent engineer with no long-term memory: that's the constraint everything else follows from. If the agent is going to be amnesiac, the project's institutional knowledge has to live in files. And those files only earn their keep if they can be trusted, found when needed, and kept current as the code moves.

We ended up with six layers, each solving a different decay problem:

Visually, the layers split into two stores — shared knowledge checked into the repo, and the agent's own local memory — feeding a single amnesiac agent at the start of every session:

The solid arrows are the read path (memory → agent at session start). The dashed arrows are the write-back and the maintenance loop that keeps the whole thing from rotting. Most teams build the read and write paths and stop there; the maintenance loop is the part that gets skipped, and it's the part that matters most.

Let's go through them.

---

Layer 1: The Memory Bank — tiered, with a reading order

The idea started as Cline's Memory Bank pattern — a structured set of Markdown files the agent reads at the start of every task to rebuild context — and we extended it from there. memory-bank/ is a directory of focused Markdown documents. The key move is that they're tiered by access frequency, and there's an explicit reading order so the agent doesn't have to read all 20 files to start work:

## Quick Start: First 4 Files to Read

1. memory-bank/ACTIVE_CONTEXT.md  - Current focus, recent changes, what's happening now
2. memory-bank/PROGRESS.md        - What's complete, what's left, current status
3. memory-bank/TECH_CONTEXT.md    - Tech stack, dependencies, environment setup
4. memory-bank/DECISIONS.md       - ADR log (why we chose what we chose)

These four files give you 90% of what you need to start working.

Everything else is a reference file, pulled in on demand and clearly labeled with when to read it:

The agent reads four files to orient, then pulls the one deep-dive doc the task actually needs, instead of paying the token cost (and the distraction cost) of loading all twenty at once. A reading order is what separates a knowledge base from a junk drawer.

ACTIVE_CONTEXT.md is deliberately the first file. It's the "where are we right now" snapshot — last quarter's big push, what's stable, what's still throwing errors:

## Current Focus

**Context-artifacts pipeline + AI pipeline reliability**

The Astro migration is long done and production is stable on that front. The work
of Apr–May 2026 has been (1) building the context-artifacts subsystem and
(2) stabilizing the LLM batch path... Those hard-won lessons are catalogued in
GOTCHAS.md — read it before touching any batch call path.

It's the paragraph a new hire would want first, and the agent is no different.

---

Layer 2: Decisions as an append-only ADR log

DECISIONS.md is an Architecture Decision Record log, with a strict format the file states up front:

ADR-style log of architectural decisions. Append new entries at the top.
Each entry: Context (what forced the choice), Decision (what we chose),
Consequence (what we now live with). Supersede — don't delete — old entries.

Two rules do most of the work. First, every entry records what forced the choice so that context is what future-you (or the agent) needs to judge whether the decision still holds. Second, you supersede rather than delete: old decisions stay, marked superseded, because the record of why we changed our minds is itself worth keeping!

A real chain from the repo:

## ADR-005: Batch LLM API for enrichment & summaries (2026-03)
Decision. Migrate enrichment + weekly summaries to a batch LLM API...

## ADR-004: Gemini Interactions API for context chaining (2025-12) — SUPERSEDED by ADR-005
Decision (original). Use Gemini Interactions API previous_interaction_id...
Consequence. Worked, but retention ceiling pushed us to ADR-005's approach. Kept for historical context.

When the agent later finds a previous_interaction_id reference lingering in old code, ADR-004 tells it the approach was intentional, why it's gone, and not to bring it back. One convention heads off a whole category of mistake: "helpfully" reviving something I already abandoned.

---

Layer 3: Gotchas (memory of what hurt)

This is my favorite layer, because it's the one most teams don't write down.

GOTCHAS.md is a catalog of failure modes in our LLM batch pipeline. Its header states the contract precisely:

Every entry here cost a production incident; the fix is in the repo, this file is the index so you don't re-pay for the discovery.

Each entry follows a Symptom / Root cause / Fix / Where structure, and — the part that earns the entry its place — explains why the obvious fix is wrong:

## Constrained-decoding runaway — raise the token floor, don't lower it

Symptom. Per-article enrichment intermittently hard-fails with finish_reason=length, even on tiny (300-char) articles, thinking already off.

Root cause. Under strict constrained JSON decoding the model occasionally
falls into a repetition/runaway loop, burning the full token budget before it
hits the cap. A retry at the same budget hits the same loop deterministically
(temperature 0.1).

Fix. Raise the maxTokens floor 8192 → 16384. Counter-intuitively, a LARGER
ceiling perturbs decoding enough to escape the loop — the identical articles
then complete cleanly in ~2,600 tokens. No steady-state cost change: billing
is per generated token, and normal completions sit far below the ceiling.

> ⚠️ When constrained decoding runs away, the fix is a HIGHER ceiling, not a
> lower one. Lowering it makes the loop fail faster, not less often.

This is also where the case for a Gotchas file — as opposed to a code comment — gets concrete. It's a fair objection that a lot of "gotchas" are really just load-bearing lines that want a comment at the call site. Some are: elsewhere in this same file is a one-line batch-engine quirk whose entire fix is enableThinking: false, and honestly a comment on that line carries most of the weight. But the runaway entry can't collapse into a comment, for two reasons:

• It isn't localized to a line. The same failure mode bit two unrelated call paths both in per-article enrichment and the artifact digest/deep-dive compiler. The fix was different at each (a token floor in one, a simpler schema in the other). The transferable knowledge is the shape of the failure, not any single constant. A comment on maxTokens = 16384 can't say "by the way, when you meet this class of bug somewhere else in the pipeline, here's how to recognize it."
• It has to reach you before you're at the keyboard. A comment only helps someone already editing the exact line. The Gotchas file is wired into the reading order — read it before any batch-path change — so the model absorbs the rule while still writing the plan.

That's the dividing line worth keeping in mind: a comment annotates a decision you've already arrived at; a gotcha intercepts the wrong decision before you make it. Lessons that are too counter-intuitive to trust at face value, or too cross-cutting to pin to one line, belong in the file — and the project wires that interception into its top-level instructions, so it's not optional.

Gotcha files are the memory of mistakes. I think they're the most valuable thing you can document.

---

Layer 4: Rules with expiry dates

This is one of the ways to try to keep context small and rules focused as you iterate. Every standing rule in CLAUDE.md carries provenance and an expiry condition. The file mandates it:

Every rule in this file should have a known origin and an expiry condition.
Format: <RULE-ID> | since: <YYYY-MM> | source: <why> | review: <YYYY-MM>.

And then there's a table:

A bare instruction can't tell you why it exists, when it started, or when to look at it again. These three fields can:

• source — why the rule exists, so you can judge whether the reason still holds.
• since — when it started, so you can tell new policy from ancient lore.
• review — when to re-examine it, so it can't quietly rot.

Some rules even encode their own graduation path: the React-island import ban says review: 2026-Q3 (promote to ESLint) — i.e., "this is a manual rule for now; by Q3 it should be a lint rule and this line should disappear." The rule knows how it's supposed to die.

And because the format is structured, rot detection is a script. scripts/harness-review-rules.sh parses the table and surfaces any rule whose review date has passed:

$ bash scripts/harness-review-rules.sh
Stale rules (review date passed):
  - investigate-services-before-trial-and-error  (review: 2026-Q1)

1 rule(s) due **for** review. Update the review column or retire the rule.

Structurally, a rule isn't a permanent fixture — it's a thing with a lifecycle that ends in either retirement or promotion to something automated:

This is what most "agent memory" setups I've used are missing. Accumulating context, plan files, rules is easy but it is hard to go back and remove stale content. Without some forcing function a rules file only grows, and every stale rule chips away at context and degrades output quality.

---

Layer 5: Progress as a verifiable audit trail

PROGRESS.md tracks in-flight and completed work — but with two rules that make it trustworthy rather than aspirational:

Discrete, verifiable items. Each item must be testable: when checked, an agent or
human can run the listed verification command (or visit the listed URL) to confirm.

Maintenance rule: when work closes, check the box, do not delete the line.
Checked items become the audit trail.

So an entry isn't a status update, it's a claim with the proof attached:

### In Flight
- [ ] Monitor LLM batch waste/retry rate post-stabilization.
      Verify: `npx tsx .../analyze-batch-retries.ts` reports retry rate < 10%.
- [ ] Monitor out-of-bounds citation-ref drop rate (~4% baseline).
      Verify: compiler logs onDroppedRefs rate is not trending up.

This buys two things. "Done" becomes falsifiable — there's a command to run, so nothing gets marked complete on a hunch. And because checked boxes stay, the file doubles as a ledger of what shipped and how it was verified. Deleting on completion would throw that history away, so the project keeps it on purpose.

---

Layer 6: Native, per-developer agent memory

The layers above are checked into the repo — shared, reviewed team knowledge. Sitting alongside them is the agent's own memory store, which is personal and local. It captures the things that are about this developer or this machine, not about the codebase.

Each memory is a single fact in its own file, with typed frontmatter:

---
name: batch-thinking-empty-content
description: Batch engine returns empty content for strict json_schema when thinking is ON
metadata:
  type: project
---

On the batch engine, the model + strict json_schema with enable_thinking: true
returns empty content ~100% of the time...
Why: batch and sync hit different inference engines.
How to apply: any compile through submitBatch with strict json_schema must set
enableThinking: false.

Memories are typed — user (who the developer is), feedback (how they want the agent to work), project (ongoing constraints), reference (external links) — and indexed in a one-line-per-fact MEMORY.md that loads each session. A feedback entry, for instance:

Don't bother the user with permission prompts for safe operations: launching
agents, reading files, running tests/lint/format, git read-only commands.
Why: user has configured broad allow-list permissions and wants an efficient flow.
How to apply: proceed on safe ops; only confirm destructive actions.

This layer answers "who am I working with and how do they like to work" so the agent doesn't have to re-derive it every session. Keeping it a separate scope from the shared Memory Bank stops one developer's preferences from leaking into the team's checked-in knowledge.

Note: the same "one fact per file, frontmatter, indexed" pattern shows up again for symbol-level code memories. Small, named, typed, indexed — the shape travels well.

---

Part 2 — The application's memory

Everything above is about the builder. Now switch hats: forget the coding agent entirely and look at Kelp the running product.

Kelp is a relatively small focused crawler that imports a few hundred sources twice a week, enriches each article, and emits weekly per-topic and per-user summaries. The naive version of that pipeline re-reads every raw article and re-synthesizes from scratch on every summary run. However, given that I'm paying for this myself, LLM calls are both expensive and as I found, actually reduce output quality.

So, I gave the application a much more structured and layered memory than the codebase. Even through it is quite different, each layer is shaped by the same overall principals.

What the product stores: the content model

The content model is a funnel. Raw crawled pages come in at the top; progressively distilled, progressively more structured knowledge comes out the bottom. Each stage stores less text and more meaning.

The central unit is source_summaries, one row per crawled piece of content, and it carries everything downstream depends on: the original content in three formats (content_html for link extraction, content_markdown for LLM processing, content_text for scoring), the AI-written summary, a primaryTopic plus a topics[] array, and AI-extracted complexity and contentType classifications. It's the memory of "what this one article is and says."

Two things hang off the article as extracted knowledge, and they're where the content model gets interesting:

• source_facts: atomic, attributable claims pulled out of an article. Each fact carries a confidence score (0–100), a citation string, and a context_snippet of the surrounding text — "cite your sources," one level deeper.
• Entities (covered next): the people, companies, technologies, and works the article is about, resolved to canonical nodes in a shared graph. This supports node traversal for people and agents.

And before any of that extraction runs, the article passes a layered dedup gate, because the single most wasteful thing a content product can do is re-read and re-reason about something it has already seen. The dedup stack escalates in cost and cleverness:

1. Exact: a SHA-256 content_hash (also title_hash, url_hash, canonical_url) catches byte-identical re-crawls.
2. Near-exact: a SimHash in content_fingerprints plus a first_paragraph_hash catch boilerplate-different-but-substantially-same pages.
3. Semantic: cosine distance over embeddings catches the same story told in completely different words (more on this below). This is super common for news related content like when Anthropic launches a new model.

Exact and near-exact are cheap string tricks. The third tier needs a model — which is the first reason embeddings exist in this system at all.

Entity extraction: from raw mentions to a clean graph

During enrichment, the LLM doesn't just summarize an article — it pulls out the entities the article is about: { name, type, confidence, context }. The type is one of a deliberately small set — company, technology, person, artist, album, song — which tells you exactly what Kelp covers (the tech world and the music world, in one taxonomy).

Extracting a name is the easy part. The part that makes this memory rather than a pile of tags — is that the same entity shows up under a dozen surface forms across thousands of articles: "OpenAI", "Open AI", "OpenAI, Inc.", "openai". Raw mentions are useless unless they collapse onto one canonical node. So every extracted mention runs a resolution ladder (EntityMatchingService.findOrCreateEntity) against the existing taxonomy of the same type, in strict priority order:

1. Exact slug match → score 100
2. Exact name, case-insensitive → 95
3. Alias match (known alternate names) → 90
4. Trigram fuzzy match, similarity > 85% → the threshold is deliberately high, because a false merge ("Anthropic" ↔ "Anthropics") corrupts the graph in a way that's expensive to unwind
5. No match → create a new canonical entity — but only if the extraction confidence clears a minimum floor. Low-confidence guesses don't get to pollute the taxonomy.

The result is a real knowledge graph, not a tag cloud:

• One canonical row per entity in entity_taxonomy, keyed by slug, linked to each article through an article_entities junction that records confidence, mention_count, and is_primary.
• entity_relationships are typed edges (works_for, founded_by, acquired_by, competes_with, parent_of) so the memory knows not just who but how they connect.
• entity_context_snippets hold per-article context about an entity ("Announced partnership with Mayo Clinic"), so a node accumulates situated facts, not just a name.
• external_ids ground each entity to canonical real-world identifiers (Wikidata QID, MusicBrainz MBID, LEI, ORCID, stock ticker). That's the entity-level version of a citation: a stable handle that survives renames and disambiguates two people with the same name.

And the graph garbage-collects itself. Duplicates that slip past the resolution ladder get cleaned up by an auto-merge pass (entity_merges), which folds a duplicate node into the canonical one and records the similarity_score and reason so an old entity ID can still be traced to where it went. Same instinct as the ADR log and the artifact versioning: supersede, don't delete. Even the knowledge graph keeps its history of changed minds.

Why embeddings: making "about the same thing" computable

Several of the behaviors above quietly depend on a question that no amount of string matching can answer: are these two things about the same thing? A title hash tells you two articles are byte-identical. Trigram similarity tells you "Anthropic" and "Anthropc" are probably the same. Neither can tell you that two articles describing the same product launch in entirely different words are the same story, or that an article about "LLM inference" is relevant to a query about "running large language models cheaply" despite sharing no keywords.

That gap is the entire reason for the embedding model. Kelp embeds content with Qwen3-Embedding-8B (4096 dimensions, served through Doubleword) into a vector space where geometric distance approximates semantic similarity. "Is this about the same thing?" becomes a distance computation — pgvector's <=> cosine operator — instead of a guess. One detail worth noting: it embeds the distilled summary + title + topics, not the raw article. The summary is already a high-quality Qwen distillation, so the vector captures the article's meaning rather than its navigation chrome and boilerplate.

That single primitive "turn meaning into geometry" underwrites four otherwise-unrelated memory behaviors:

• Semantic dedup — the third tier of the dedup gate above: cosine similarity catches the same story told in different words, so the pipeline never pays to summarize it twice.
• Recall / retrieval — the artifact retriever's hybrid search (detailed below), which surfaces relevant remembered context even when the prompt and the memory share no literal keywords.
• Theme matching — when keyword matching fails to attach an article to a weekly-summary theme, semantic similarity to the theme name is the fallback.
• Recommendations — average the embeddings of a theme's articles into a centroid, then rank candidate articles by cosine distance to it.

A memory is only as good as its ability to recognize "I've seen this before" and "this is relevant to what I'm doing now" — both semantic-sameness questions. Hashes give you identity; embeddings give you similarity. That's why the embedding model is load-bearing, not a bolt-on.

The synthesized layer: context-artifacts

On top of the stored content and the extracted graph sits the part we actually call the application's memory: the context-artifacts subsystem. The goal is simply to move expensive synthesis upstream to compile once per article/topic, reuse many times and attach citations so downstream summaries are grounded and auditable.

What it remembers

A context artifact is one pre-compiled, citation-grounded summary, uniquely keyed by a tuple:

(taskType, subjectKind, subjectId, personaId)

• brief: distilled from a single article (the cheap, high-volume path)
• digest / deep_dive: synthesized across the N recent articles in a topic

Instead of feeding raw articles into the weekly-summary prompt, the pipeline feeds these pre-distilled briefs. The application remembers what it already concluded about each article, so it never has to reason it out again.

The flow — synthesize once on the write side, recall many times on the read side — mirrors the builder's read/write loop almost exactly, and is quite low-cost for me to run. Four properties make it trustworthy, and each one rhymes with a layer from Part 1.

How an artifact is written: version, don't mutate

A new artifact is written as a new versioned row that supersedes the prior one, never an in-place update. The schema enforces it with a partial unique index that allows exactly one current (non-superseded) row per subject. So the application's memory is append-only history: you can always see what an earlier compiler version concluded, and new knowledge supersedes old rather than overwriting it.

How it knows when it's stale

The builder's rules expire on a review date. An artifact expires on its inputs. The compile key carries two staleness signals:

• inputHash — changes when the underlying source summaries change.
• compilerVersion — each prompt module exports its own *_COMPILER_VERSION; bump it and every artifact built by the old prompt is now considered stale and gets recompiled.

That compilerVersion field is the production analogue of "this rule is due for review" — a deliberate cache-invalidation lever: change how you think, bump the version, and the memory rebuilds itself. Memory that knows when it's out of date, enforced by a hash instead of a shell script.

How every claim cites its source

Every artifact carries a citations[] array, and every field in the output points back into it by index. At read time, those indices are hydrated against the source_summaries table so each claim resolves to a real URL and title:

// retrieve.ts — join each artifact's citations[] with a source_summaries lookup,
// preserving the original index so per-field citation refs still resolve.
export function hydrateCitations(
  raw: ArtifactCitation[],
  byId: Map<string, { url: string; title: string }>
): HydratedCitation[] {
  return raw.map((c, index) => ({
    index,
    sourceSummaryId: c.sourceSummaryId,
    url: byId.get(c.sourceSummaryId)?.url ?? '',
    title: byId.get(c.sourceSummaryId)?.title ?? '',
    ...(c.note ? { note: c.note } : {}),
  }));
}

The read path enforces a strict no-dangling-ref invariant, and validates every stored row against its Zod schema before trusting it — because a memory you can't trust is worse than no memory:

// A corrupt row with [null] or wrong-type ids would produce garbage hydration
// downstream — so validate at read time and skip what doesn't parse.
output = parseArtifactOutput(row.task_type, row.output, citations);

Don't blindly trust what's written down; verify it still holds — the same skepticism applied to a stale rule, expressed in code.

How retrieval is budgeted

Just as the Memory Bank is tiered so the agent doesn't drown in context, the artifact retriever caps how much memory loads into the summary prompt (by both count and token budget). This is something I might change with infinite money though:

// Greedily accumulate hits until either budget is exhausted. The first hit is
// always admitted even if it alone exceeds maxTokens — otherwise a single large
// artifact would silently produce an empty result.
export function applyBudget(hits, budget) {
  const kept = [];
  let total = 0;
  for (const hit of hits) {
    if (kept.length >= budget.maxArtifacts) break;
    if (kept.length > 0 && total + hit.approxTokens > budget.maxTokens) break;
    kept.push(hit);
    total += hit.approxTokens;
  }
  return { kept, totalApproxTokens: total };
}

And when the summary needs to find relevant memory rather than look it up by id, it runs hybrid retrieval using pgvector semantic search and Postgres full-text search, fused with Reciprocal Rank Fusion over only the current, non-superseded rows. Databases are awesome.

It's a recall system: surface the most relevant remembered context, within budget, and leave the rest on disk.

The reliability tax

All of this, though not like super novel or anything, was challenging to get working well. It feels like I spent 95% of my time getting high quality data and only 5% of my time building a product I wanted to use.

Giving the application a memory opened up a whole new failure surface, and it's the most-documented part of the codebase for that reason. Compiling artifacts through an LLM batch engine under strict json_schema has failed in…many ways. The LLM tools are not very mature and change a ton. Minor package update and minor model version bumps caused empty content, truncation, out-of-bounds citation refs (each one is catalogued in GOTCHAS.md!). So the two systems don't just run in parallel, they feed each other: the application's memory broke in production, and those lessons became entries in the builder's memory so I wouldn't pay for those mistakes again. Still, it was a lot of iteration.

---

The meta-pattern: every layer fights a different kind of rot

Put the two systems side by side and the interesting thing shows up. The builder's memory and the application's memory were built for different reasons, one in Markdown and one in Postgres but landed on the same principles anyway. That they converged is the best evidence I have that these are real engineering patterns:

Within each system, every layer is in turn shaped by how its kind of memory goes bad:

• Append-only history (ADRs, progress): never delete, only supersede or check off — because why we changed our minds is itself the memory.
• Negative knowledge (gotchas): write down what hurt, and especially why the obvious fix is wrong, planted as a tripwire where the next person will trip.
• Expiry as a first-class field (rule provenance): every standing rule carries a review date, and a script enforces it — so the rules file can't silently rot.
• Verifiable claims (progress): "done" ships with the command that proves it.
• Tiering (memory bank): a fixed reading order plus on-demand reference docs, so orientation is cheap and depth is available.
• Scope separation (shared repo memory vs. native per-dev memory): team knowledge and personal preference live in different stores.

In both cases, there's even an end-of-session sweep. npm run harness:clean reports working-tree drift, large untracked files, and recent agent actions before you walk away vs the application which has nightly cleanup tasks. Both are the memory equivalent of closing the books at the end of the day so tomorrow starts clean.

---

What about buying this off the shelf?

A fair question hangs over all of this: there's now a whole category of products that sell "memory for AI." Why hand-roll it?

The honest answer is that the off-the-shelf systems solve one of the six principles (recall) extremely well, and leave the other five to you. That's a fine trade if retrieval is the only thing you were missing or you have a very large team.

For the application's memory, the closest commercial analogue is the new class of managed agent-grounding layers: Microsoft's Foundry IQ, AWS Bedrock Knowledge Bases, Google Vertex AI Search. They're genuinely great. Foundry IQ, for instance, is a managed multi-source knowledge base built on Azure AI Search that does agentic retrieval which decomposes a question into parallel subqueries, reranks, and synthesizes an answer with citations. That's a more sophisticated recall path than Kelp's single-shot hybrid search, and if recall were the whole problem I'd have reached for it.

But measure one against the six principles and the gap is the whole point:

These systems index documents; they don't version conclusions. A managed knowledge base will happily retrieve your source material with citations. What it won't do is the part this whole post is about is compile a synthesized artifact once, supersede the prior version in a transaction, mark it stale when its inputHash or compilerVersion changes, and keep the old versions as history. Versioning, staleness, and garbage collection aren't retrieval features; they're memory features, and they live in the domain model, not the vector index. The ❌ rows above aren't some missing feature, they are simply out of scope. A retrieval layer rents you recall; the synthesis-memory around it is still yours to build. Building something similar for a Doctor might look quite different from what I did.

Codebases also have their own off-the-shelf shelf tools: agent-memory frameworks like Mem0, Letta (formerly MemGPT), and Zep. These auto-extract facts from conversations into a vector store and recall them later. The systems are great for teams where you want to accumulate memory automatically or projects that are an order of magnitude larger than Kelp where a database is more appropriate than markdown files. I imagine one could layer on a 'garbage-collection' process and keep the memory high quality. For me though, The hard-won lesson is that forgetting on purpose (expiry dates, supersession, the end-of-session sweep) is what keeps a memory trustworthy and as Kelp scales, I may use those tools with my custom 'forgetting' layer on top.

None of this is a knock on those products. The systems you can buy are mostly recall engines. The half that's actually hard to ensure quality, reliable LLM output is knowing what to keep, what to supersede, and when to throw something out. That is still yours to build, whichever retrieval layer you rent.

---

Takeaways

Whether you're giving a coding agent a usable memory or building a product that remembers what it has read, the lessons from doing both in one codebase turned out to be the same:

1. Treat memory as a subsystem with tiers, not one big file. A reading order and on-demand reference docs beat a 4,000-line context dump.
2. Write down what hurt. Gotcha/incident files are the highest-ROI memory you can keep — and always explain why the obvious fix is wrong.
3. Record the why, and supersede instead of deleting. The history of changed minds prevents re-litigating settled decisions and resurrecting dead ones.
4. Give every rule an expiry date, and automate the staleness check. Accumulating context is easy; expiring it is the hard part nobody does. A review date plus a 60-line shell script turns "is this still true?" into a date comparison.
5. Make "done" verifiable. A progress item without a verification command is a wish, not a record.

The model will keep waking up with amnesia, and the product will keep facing more content than it can read. You can't fix the forgetting in either case. What you can do is leave behind a filing system good enough that an amnesiac expert is productive, and a summary pipeline can build on last week's conclusions instead of redoing them. A filing system nobody prunes is just a landfill, which is why the part that earns its keep is the part that throws things out, marks them superseded, or rebuilds them when the inputs change.

Two systems, built separately, fell into the same principles. That's the part I'd keep.

Also, I just have to say: databases are awesome. The thing I'd ask is - 'how can I put as much information my agents need into a database' and how can I use row and table permissions to formally restrict access to only what the agent need? Could the project memory be in a database as well?