In April, Andrej Karpathy published a 12 KB gist describing how he keeps a personal knowledge base: an LLM agent maintains a folder of interlinked markdown, and he just curates sources and asks questions. It pulled about 16 million views and, as of this week, 1,061 comments.
I captured every comment and worked through the corpus, because I have this problem twice. A personal knowledge base an agent maintains for me daily. And a company knowledge base where agents had started answering with confident nonsense.
The gist's claim is easy to state. RAG retrieves from raw documents on every query, so nothing accumulates. Let the agent maintain a persistent wiki instead: ingest compiles knowledge once, queries hit the compiled layer, a periodic lint keeps it healthy. Humans abandon wikis because maintenance grows faster than value; agents don't get bored.
He's right, and I want that on the record before the rest of this post. I run this pattern every day and the compounding is real. Ask something that touches five sources and the cross-references are already there.
The comments are where it gets interesting. Past the thank-yous and the product pitches, the thread reads like a field report from everyone who pushed the pattern until it broke. The same failures keep coming back, and they cluster into four families:
Fourteen distinct failure modes in total. Here is each family, with the evidence behind it.
A hallucination resets when the conversation ends. A wrong note does not.
One commenter who ran the pattern in production for 12 months put it precisely: a bad wiki article becomes a prior that poisons future generations. The agent writes a plausible summary, the summary feeds the next answer, the next answer gets filed as a new page. Now two pages cite each other as evidence.
Multi-agent setups make it worse. In a Show HN thread for one gist-inspired tool, maintainers reported agents citing other agents' unverified entries, and one commenter summarized the dynamic in six words: everyone is writing, nobody is reading.
The fix that thread after thread converged on is a gate. The agent drafts freely, a human promotes, and unpromoted drafts don't count as evidence for other pages. Slower, yes. But in a durable store, garbage in is not garbage out. It's garbage compounding.
My favorite comment in the corpus is a quiet one. Someone ran the pattern
for a few months on notes about apps they maintain. A note said a
dashboard required admin access. The permission check had returned true
for everyone, always.
Nothing in the vault contradicted the note, so every lint pass blessed it. The contradiction wasn't between notes. It was between a note and a line of code the notes had never read.
This is the failure class most audits miss: supersession that happens outside the knowledge base. Prices change, APIs change, rules change, and the summary that was true in March serves confidently in August. The same family holds the smaller rots: facts silently overwritten instead of visibly superseded, index files drifting from the pages on disk, orphan pages nothing links to, numbers with no pointer back to their source.
The fix is boring and works. Claims about anything outside the KB carry a pointer to what they describe and a date when it was checked, and the lint pass re-verifies the oldest ones. Your notes can't see the world change. Give them timestamps so at least you can see them age.
The phrase I keep coming back to from my own company mess: garbage in, garbage out, but garbage is relative to the task. The competitor analysis that is signal for a pricing decision is noise inside a debugging session. Feed an agent both and you don't get a smarter agent, you get a distracted one.
The evidence here is solid by now. Chroma measured 18 frontier models degrading as input grows, even on trivial tasks. Drew Breunig documented a small model that fails a benchmark with 46 tools available and passes with 19. The problem wasn't capability. It was the clutter.
The working consensus, in the comments and in the enterprise patterns alike, is the same move: scope, don't filter. Separate corpora per domain or task, routed by what the task needs. Load the 19 tools, not all 46.
Almost every architecture decision in the gist quietly assumes a single human with a single agent.
The most instructive comment in the corpus is a production report from someone running 6 specialized agents with 50+ subagents against a shared knowledge base. To keep it alive they built 13 extensions: five separate wikis with five different permission models, file-based capability tokens, and a verify-before-assert hook, because one wrong assertion compounds through the whole dispatch chain.
Enterprises hit the same wall with more money on the line. A Gartner survey had roughly 40% of Microsoft 365 Copilot rollouts delayed over oversharing, because the agent faithfully retrieved everything employees technically had access to.
Plain markdown has no answer to permissions. Everyone who needs them either splits the corpus by audience or moves that part to permission-aware retrieval, where access is filtered before the model sees context, never after.
The research forced my hand at home. My KB now runs:
None of this makes rot impossible. It makes rot visible.
I distilled all of it into a small free kit: an audit prompt you paste into Claude Code, Codex, or Cursor that checks a knowledge base against all 14 failure modes read-only and returns a report card, plus the modes with fixes and the templates that prevent them. Get the kit here.
The gist ends by saying the LLM finally solves the Memex's maintenance problem. After 1,061 comments, I'd sharpen it: agents solve the maintenance labor problem and inherit the maintenance judgment problem, and judgment is the part you can't skip. What excites me is how cheap good judgment has become to encode: a gate here, a timestamp there, a lint prompt on a schedule. I'm excited for agents whose memory gets more trustworthy the longer you work with them, because the notes underneath finally stay clean.
Occasional notes on software, tools, and things I learn. No spam.
Unsubscribe anytime.