HNSW vs IVFFlat: choosing and building your pgvector index

Up to a few hundred thousand rows, an exact scan over a vector column is genuinely fast enough. You don't need an ANN index. The query plan is a sequential scan, every row gets compared, the answer is exactly right.

Past that point, you need an index. Which one is a real decision, not a shrug. HNSW and IVFFlat have different build times, different memory profiles, different recall behavior, and different operational stories. Pick wrong and you'll either rebuild every week or live with results that quietly miss.

This post is the choice. Part 5 is what you do with the knobs after.

The two options, in one sentence each

HNSW builds a layered graph and walks it at query time. Slower to build, fast to query, high recall at default settings, holds the whole graph in memory.

IVFFlat divides the vector space into cells and probes a subset at query time. Faster to build, cheaper memory, recall is more sensitive to the data shape and the tuning.

Most production pgvector deployments end up on HNSW. That's not a fashion trend, it's because the speed-recall tradeoff is genuinely better for most workloads and the operational pain (slow build, big memory footprint) is predictable. The community rule of thumb that holds up in production: under roughly 10M vectors, standard HNSW is fine. Above that you start having opinions about quantization and graph parameters. IVFFlat is the better choice when you have either very strict build times, very tight memory, or a query pattern that wants large result sets. I wrote a shorter overview comparing the two that's worth a skim before this post if you've never used either.

The animation is worth a longer look before you commit:

What you're watching matters. HNSW walks a small number of edges to reach a tight neighborhood; IVFFlat picks the right cells and then exhaustively checks them. The shapes of "wrong" are different. HNSW with too-low ef_search misses neighbors that live on the other side of the graph. IVFFlat with too-few probes misses neighbors that fell into a cell you didn't check.

The decision, made on real constraints

I'll skip the "it depends" answer. Here's how I'd actually pick.

Pick HNSW if any of these are true. Your write rate is moderate (under a few hundred inserts per second sustained). Your latency budget is under 50ms p95. Your recall target is above 90%. You can afford a one-time slow build. Memory headroom on the box is at least 1.5x the index size.

Pick IVFFlat if any of these are true. Your build budget is small and you re-index often. Memory is tight (you can't fit the HNSW graph in RAM comfortably). You query for large result sets (500+ rows). You're willing to spend more time tuning probes per query type than tuning ef_search.

The default for this series is HNSW. The docs assistant is read-heavy, the latency budget is interactive, and we want the recall headroom for Part 7's evals to be meaningful. Almost everything from here on assumes HNSW unless called out.

Build the index without locking the table

This is the part that bites people. A naive CREATE INDEX takes an ACCESS EXCLUSIVE lock for the whole build. On a real table that means your writes are blocked for however long the build takes. On a 200k-row table, that's a few minutes. On a 5M-row table, it can be hours.

You almost always want CONCURRENTLY. Drizzle Kit will generate a normal CREATE INDEX from the schema; you write the concurrent build by hand:

CREATE INDEX CONCURRENTLY chunks_embedding_hnsw_idx
ON chunks
USING hnsw (embedding vector_cosine_ops);

A few things to know about CONCURRENTLY before you ship it:

1. It can't run inside a transaction. Most migration tools wrap each migration in BEGIN ... COMMIT. You either disable the wrapper for this migration or run the index build outside the migration pipeline.

2. If it fails, the index is left INVALID. Drop it (DROP INDEX CONCURRENTLY chunks_embedding_hnsw_idx;) and try again. An invalid index does nothing for queries and slows down writes.

3. It still scans the whole table. "Concurrently" means "without blocking writes," not "in the background while you do other things." It's the same amount of work, scheduled politely.

Knobs that decide how long the build takes

Two settings matter for build speed, and they're the ones almost everyone leaves at default.

-- Bigger here = faster build at the cost of memory during the build.
SET maintenance_work_mem = '2GB';
 
-- Use multiple workers for the build. Default is often 2.
SET max_parallel_maintenance_workers = 7;

On a 1M-row table with 1024-dim vectors, raising maintenance_work_mem from the default 64MB to 2GB and max_parallel_maintenance_workers from 2 to 7 has cut my build times from an hour-plus to under ten minutes. The numbers will vary on your hardware; the direction won't.

Two HNSW-specific build parameters live on the index itself:

CREATE INDEX CONCURRENTLY chunks_embedding_hnsw_idx
ON chunks
USING hnsw (embedding vector_cosine_ops)
WITH (m = 16, ef_construction = 64);

m is the max connections per node in the graph (default 16). Higher means a denser graph, which improves recall but inflates the index size and build time. 16 is fine for most workloads; raise to 24 or 32 for high-recall use cases. ef_construction is the candidate-list size during build (default 64). Higher improves the quality of the graph but slows the build. 64 to 200 covers most cases. These are build-time choices and you can't change them without rebuilding, so pick once and move on.

Build it on data that exists

A common mistake worth naming: people build the index on an empty table, then ingest. With HNSW that's fine (the index supports incremental inserts well), but you miss the chance to build with parallel workers and maintenance_work_mem cranked, because there's nothing to build from.

The order I'd recommend for the project:

1. Ingest the corpus into the table with no ANN index. Inserts are fast, no index to maintain. 2. Bump maintenance_work_mem and max_parallel_maintenance_workers for the session. 3. Build the index CONCURRENTLY. Walk away. Come back to a query plan that uses it.

For subsequent ingestions (a new doc batch every day, say), let the index maintain itself. The amortized cost per insert with HNSW is small.

Confirm the index is used with EXPLAIN

EXPLAIN answers this directly. After the index is built:

EXPLAIN (ANALYZE, BUFFERS)
SELECT id, body
FROM chunks
ORDER BY embedding <=> '[0.01, 0.02, ...]'
LIMIT 10;

Look for an Index Scan using chunks_embedding_hnsw_idx. If you see a Seq Scan instead, the planner thinks a sequential scan is cheaper. Two common reasons: the table is too small (Postgres knows it can scan it faster than it can walk the graph) or the operator in your query doesn't match the operator class on the index. If you built with vector_cosine_ops and query with <-> (L2 distance), the index isn't applicable. We dig much deeper into reading these plans in Part 5.

From the trenches

Don't drop and rebuild "to refresh" the index. HNSW indexes don't degrade with time; they degrade with very high deletion rates because deleted graph nodes still exist until vacuum cleans them up. If you're worried, run VACUUM first. Drop-and-rebuild is an hour you didn't need to spend.

Wrap the build in idle_in_transaction_session_timeout = 0. Long builds against a connection pooler can be killed mid-build by aggressive timeouts. The build then leaves an INVALID index and you start over.

Half-precision vectors are tempting. pgvector supports halfvec for indexes, which halves memory at a small recall cost. For 1024-dim vectors at 5M rows it's a meaningful difference. Don't reach for it on day one. Get the full-precision index running, get the eval working, then measure the recall delta with halfvec on your data before committing.

Build on a replica, then promote. For very large tables, the safest path is to build on a streaming replica (max_parallel_maintenance_workers cranked, no traffic), then either replicate the new index back or promote the replica to primary. This avoids touching production for the hours of the build entirely. We come back to this pattern in Part 8 for re-embedding migrations.

The index is built, but not tuned

You now have an HNSW index built CONCURRENTLY, on a table that already holds data, with build parameters chosen on purpose. You also know the conditions under which IVFFlat would have been the better call.

What you don't control yet is the speed-recall curve at query time. The index will give you reasonable results at default settings, and reasonable is not the goal of this series. Part 5 is the tuning.