Online

If you're working with vector embeddings in PostgreSQL, you've probably heard of pgvector. It's an amazing extension that brings vector similarity search capabilities to Postgres. One of the key features of pgvector is its support for different indexing options to speed up your queries. Today, we're going to dive into these options and help you choose the right one for your use case.

pgvector offers two main indexing methods: HNSW (Hierarchical Navigable Small World) and IVFFlat (Inverted File Flat). Each has its own strengths and trade-offs, so let's break them down.

HNSW: The Speed Demon

HNSW is the newer kid on the block, introduced in pgvector 0.5.0. It's designed for lightning-fast queries and generally offers better performance than IVFFlat in terms of the speed-recall trade-off. Here's what you need to know:

It creates a multilayer graph structure for efficient searching.
HNSW indexes can be created even when your table is empty, which is super convenient.
It supports vectors up to 2,000 dimensions, or 4,000 if you're using half-precision.

To create an HNSW index, you'd use something like this:

CREATE INDEX ON items USING hnsw (embedding vector_l2_ops);

You can tune HNSW performance with two main parameters:

m: The max number of connections per layer (default is 16)
ef_construction: The size of the dynamic candidate list for constructing the graph (default is 64)

For example:

CREATE INDEX ON items USING hnsw (embedding vector_l2_ops) WITH (m = 16, ef_construction = 64);

During queries, you can adjust the ef_search parameter to balance between speed and recall:

SET hnsw.ef_search = 100;

IVFFlat: The Reliable Workhorse

IVFFlat has been around longer and offers a different approach:

It divides vectors into lists and searches a subset of those lists.
It has faster build times and uses less memory than HNSW.
It's better suited for scenarios where you need a large number of results (500+).

To create an IVFFlat index:

CREATE INDEX ON items USING ivfflat (embedding vector_l2_ops) WITH (lists = 100);

The key to good performance with IVFFlat is choosing the right number of lists. A good rule of thumb is:

For up to 1M rows: use rows / 1000
For over 1M rows: use sqrt(rows)

During queries, you can adjust the number of probes to balance between speed and recall:

SET ivfflat.probes = 10;

Choosing Between HNSW and IVFFlat

So, which one should you use? Here are some guidelines:

If you need blazing-fast queries and can afford longer index build times, go with HNSW.
If you're dealing with a large number of vectors and need faster index creation, IVFFlat might be a better choice.
For queries that return a large number of results (500+), IVFFlat is generally more suitable.
If you're working with an empty or small table that will grow over time, HNSW can be a good option since it doesn't require data to create the index.

Remember, you can always create both types of indexes and compare their performance for your specific use case.

Bonus Tip: Half-Precision Indexing

If you're working with high-dimensional vectors (up to 4,000 dimensions), you can use half-precision indexing to save space and potentially improve performance:

CREATE INDEX ON items USING hnsw ((embedding::halfvec(3)) halfvec_l2_ops);

This creates an index using half-precision floats, which can be particularly useful for large datasets.

tl;dr

pgvector offers two main indexing options: HNSW and IVFFlat. HNSW is faster for queries but slower to build, while IVFFlat is quicker to build and better for large result sets. Choose HNSW for speed and IVFFlat for flexibility with large datasets. Don't forget to experiment with parameters like ef_search for HNSW and probes for IVFFlat to fine-tune performance.