---
title: "I read all 1,061 comments on Karpathy's llm-wiki gist"
description: "Karpathy's llm-wiki gist named the agent-maintained markdown wiki pattern. Its 1,061 comments named how it fails, and this post groups them into four patterns."
canonical: https://omiid.me/notebook/66/i-read-all-1-061-comments-on-karpathy-s-llm-wiki-gist
author: Omid Sayfun
published: 2026-08-17
updated: 2026-08-18
---

# I read all 1,061 comments on Karpathy's llm-wiki gist

In April, Andrej Karpathy published a
[12 KB gist](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f)
describing how he keeps a personal knowledge base: an LLM agent maintains a
folder of interlinked markdown, and he just curates sources and asks
questions. It pulled about **16 million views** and, as of this week,
**1,061 comments**.

I captured every comment and worked through the corpus, because I have this
problem twice. A personal knowledge base an agent maintains for me daily.
And a company knowledge base where agents had started answering with
confident nonsense.

The gist's claim is easy to state. RAG retrieves from raw documents on
every query, so nothing accumulates. Let the agent maintain a persistent
wiki instead: ingest compiles knowledge once, queries hit the compiled
layer, a periodic lint keeps it healthy. Humans abandon wikis because
maintenance grows faster than value; agents don't get bored.

He's right, and I want that on the record before the rest of this post. I
run this pattern every day and the compounding is real. Ask something that
touches five sources and the cross-references are _already there_.

The comments are where it gets interesting. Past the thank-yous and the
product pitches, the thread reads like a field report from everyone who
pushed the pattern until it broke. **The same failures keep coming back,
and they cluster into four families:**

1. **Garbage compounds**: wrong entries become priors, summaries lose
   their sources, everything writes and nothing reads.
2. **Notes rot against reality**: silent supersession, index drift,
   orphan pages, claims that outlive the systems they describe.
3. **Relevance is task-relative**: in one big corpus, every task
   inherits the noise of all the others.
4. **The pattern assumes one of you**: no permissions, no concurrency,
   no sensitivity split.

Fourteen distinct failure modes in total. Here is each family, with the
evidence behind it.

## 1. Garbage compounds

A hallucination resets when the conversation ends. **A wrong note does
not.**

[One commenter](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f)
who ran the pattern in production for **12 months** put it
precisely: a bad wiki article becomes a prior that poisons future
generations. The agent writes a plausible summary, the summary feeds the
next answer, the next answer gets filed as a new page. Now two pages cite
each other as evidence.

Multi-agent setups make it worse. In a
[Show HN thread](https://news.ycombinator.com/item?id=47899844) for one
gist-inspired tool, maintainers reported agents citing other agents'
unverified entries, and
[one commenter](https://news.ycombinator.com/item?id=47900508) summarized
the dynamic in six words: everyone is writing, nobody is reading.

The fix that thread after thread converged on is a gate. The agent drafts
freely, a human promotes, and unpromoted drafts don't count as evidence
for other pages. Slower, yes. But in a durable store, garbage in is not
garbage out. It's garbage compounding.

## 2. Notes rot against reality

My favorite comment in the corpus is a quiet one. Someone ran the pattern
for a few months on notes about apps they maintain. A note said a
dashboard required admin access. The permission check had returned `true`
for everyone, always.

Nothing in the vault contradicted the note, so every lint pass blessed
it. **The contradiction wasn't between notes. It was between a note and a
line of code the notes had never read.**

Most audits miss this failure class: supersession that happens
_outside_ the knowledge base. Prices change, APIs change, rules change,
and the summary that was true in March serves confidently in August. The
same family holds the smaller failures: facts silently overwritten instead of
visibly superseded, index files drifting from the pages on disk, orphan
pages nothing links to, numbers with no pointer back to their source.

The fix is simple. Claims about anything outside the KB carry a
pointer to what they describe and a date when it was checked, and the
lint pass re-verifies the oldest ones. Your notes can't see the world
change. Give them timestamps so at least you can see them age.

## 3. Relevance is task-relative

The phrase I keep coming back to from my own company mess: garbage in,
garbage out, but **garbage is relative to the task**. The competitor
analysis that is signal for a pricing decision is noise inside a
debugging session. Feed an agent both and you don't get a smarter agent,
you get a distracted one.

The evidence here is solid by now.
[Chroma measured 18 frontier models](https://research.trychroma.com/context-rot)
degrading as input grows, even on trivial tasks.
[Drew Breunig documented](https://www.dbreunig.com/2025/06/22/how-contexts-fail-and-how-to-fix-them.html)
a small model that fails a benchmark with **46 tools** available and
passes with **19**. The problem wasn't capability. It was the clutter.

The working consensus, in the comments and in the enterprise patterns
alike, is the same move: scope, don't filter. Separate corpora per domain
or task, routed by what the task needs. Load the 19 tools, not all 46.

## 4. The pattern assumes there is one of you

Almost every architecture decision in the gist assumes a single
human with a single agent.

The most instructive comment in the corpus is a
[production report](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f)
from someone running **6 specialized agents** with **50+ subagents**
against a shared knowledge base. To keep it alive they built 13 extensions:
five separate wikis with five different permission models, file-based
capability tokens, and a verify-before-assert hook, because one wrong
assertion compounds through the whole dispatch chain.

Enterprises hit the same wall with more money on the line.
[A Gartner survey](https://www.computerworld.com/article/3542000/microsoft-365-copilot-rollouts-slowed-by-data-security-roi-concerns.html)
had roughly **40%** of Microsoft 365 Copilot rollouts delayed over
oversharing, because the agent faithfully retrieved everything employees
technically had access to.

**Plain markdown has no answer to permissions.** Everyone who needs them
either splits the corpus by audience or moves that part to
permission-aware retrieval, where access is filtered _before_ the model
sees context, never after.

## What I changed in my own knowledge base

The research forced my hand at home. My KB now runs:

- a hard word cap on its map file (a map, not a journal),
- supersession callouts instead of silent edits,
- a scheduled lint pass every other day,
- and after that permission-check comment, pointers plus dates on every
  claim about external systems.

None of this makes rot impossible. It makes rot visible.

I distilled all of it into a small free kit: an audit prompt you paste
into Claude Code, Codex, or Cursor that checks a knowledge base against
all 14 failure modes read-only and returns a report card, plus
[the modes with fixes](https://omiid.me/notebook/67/your-agent-s-knowledge-base-is-lying-to-you-run-these-14-checks)
and the templates that prevent them.
[Get the kit here](https://omiid.me/kb-audit?utm_source=blog&utm_content=post-66).

The gist ends by saying the LLM finally solves the Memex's maintenance
problem. After 1,061 comments, I'd sharpen it: agents solve the
maintenance _labor_ problem and inherit the maintenance _judgment_
problem, and judgment is the part you can't skip. What excites me is how
cheap good judgment has become to encode: a gate here, a timestamp there,
a lint prompt on a schedule. I'm excited for agents whose memory gets
more trustworthy the longer you work with them, because the notes
underneath finally stay clean.
