---
title: "Search text and screenshots with one pgvector column"
description: "One Postgres table can hold text and screenshot embeddings in the same vector column. Set up the schema, the Voyage embedding call, and the first query."
canonical: https://omiid.me/notebook/58/search-text-and-screenshots-with-one-pgvector-column
author: Omid Sayfun
published: 2026-08-03
updated: 2026-08-03
---

# Search text and screenshots with one pgvector column

By the end of this post, the project works end to end: a Postgres you
self-host, a Drizzle schema, a Voyage call that takes both text and an
image, a table with a few hundred rows of real Postgres docs and
screenshots, and one query that returns the right thing.

That sounds small. The reason it's a whole post is that almost every later
decision (indexing, tuning, evals, re-embedding) is shaped by what you pick
in the next twenty minutes. The dimension on your vector column, the
extension version, the build flags on Postgres, the way you batch embedding
calls. None of these throw if you get them wrong. They just make _Part 5_
miserable.

So we set up like the next six posts exist.

## The project: a docs assistant for text and screenshots

The project is a multimodal docs assistant. The corpus is **Postgres'
own documentation pages plus a curated set of screenshots** (psql output,
EXPLAIN plans, pgAdmin views). Real prose, real images, the kind of thing
your users would search.

We're going to want this assistant to answer questions like "how do I
recover from a failed `CREATE INDEX CONCURRENTLY`?" and have it pull the
right doc paragraph _and_ the screenshot of the relevant error message. Same
query, one index, both modalities.

## Run Postgres on your own machine

Self-hosted is a deliberate choice. We need to see the build flags, the
config knobs, and what changing `shared_buffers` does. You can't
see those on a managed Postgres without indirection, and Parts 4 and 5
need them in the foreground. If you eventually run on Supabase or RDS, the
intuition transfers. The opposite direction (start managed, learn to tune)
mostly doesn't.

Use the official `pgvector` image so the extension is already in the binary:

```bash
# docker-compose.yml
services:
  db:
    image: pgvector/pgvector:pg16
    environment:
      POSTGRES_PASSWORD: dev
      POSTGRES_DB: docs_assistant
    ports:
      - "5432:5432"
    volumes:
      - pgdata:/var/lib/postgresql/data
volumes:
  pgdata:
```

Save that as `docker-compose.yml` and start the container:

```bash
docker compose up -d
```

Then enable the extension once. Drizzle Kit won't do this for you, and a fresh
database fails without it:

```sql
CREATE EXTENSION IF NOT EXISTS vector;
```

If this is your first time wiring pgvector into Drizzle, I wrote the
[setup walkthrough](https://omiid.me/notebook/15/how-to-store-vector-embeddings-in-postgres-with-drizzle-orm)
separately. The piece below assumes you've got `drizzle-orm`, `drizzle-kit`,
and `postgres` (or `pg`) installed.

## The schema: one table, one vector column

One table for now. Splitting "documents" and "chunks" gets pulled out in
Part 3 when chunking earns it; right now a single table keeps the cursor
on the parts that matter.

```ts
// src/db/schema.ts
import {
  pgTable,
  serial,
  text,
  jsonb,
  vector,
  index,
} from "drizzle-orm/pg-core";

export const chunks = pgTable(
  "chunks",
  {
    id: serial("id").primaryKey(),
    source: text("source").notNull(),
    body: text("body"),
    // voyage-multimodal-3.5 returns 1024-dimensional vectors by default.
    // You can request 256, 512, or 2048 instead. Whatever you pick, every
    // row has to agree on it.
    embedding: vector("embedding", { dimensions: 1024 }),
    metadata: jsonb("metadata").$type<{
      kind: "text" | "image";
      url?: string;
      pageId?: string;
      chunkIndex?: number;
    }>(),
  },
  (t) => [
    index("chunks_embedding_hnsw_idx").using(
      "hnsw",
      t.embedding.op("vector_cosine_ops"),
    ),
  ],
);
```

A few notes worth keeping in your head, because they all get cashed in
later:

1\. **`dimensions: 1024` is a one-way door.** Every row has to agree on it.
You can re-embed the whole table, sure, but that's Part 8 with a careful
zero-downtime story. Pick on purpose.

2\. **The HNSW index uses `vector_cosine_ops`.** That has to match the
operator you query with (`<=>`). If you query with `<->` instead, the index
goes unused and you'll wonder why everything is slow. I went deeper on
[what each pgvector operator actually means](https://omiid.me/notebook/32/what-the-pgvector-operators-and-actually-mean)
in a separate post.

3\. **One column, both modalities.** `body` is nullable because image rows
don't have text content. The `metadata.kind` field tells you which it is
when reading back results.

Run `drizzle-kit generate` and `drizzle-kit migrate` and you have the table.
Don't forget the `CREATE EXTENSION` above the first migration if this is a
brand new database.

## Voyage embeds text and images through one call

Voyage's multimodal endpoint takes a list of inputs, each input is a list of
content pieces, and each piece is either text or an image reference. The
shape is simple on purpose: text and image go through the _same_ call, and
each input returns one vector in the same coordinate space.

There's an SDK but I prefer `fetch` for code that's going to be in front of
people. It's verifiable against the docs without grep:

```ts
// src/embed.ts
type VoyagePiece =
  | { type: "text"; text: string }
  | { type: "image_url"; image_url: string }
  | { type: "image_base64"; image_base64: string };

export async function embed(
  inputs: VoyagePiece[][],
): Promise<number[][]> {
  const res = await fetch(
    "https://api.voyageai.com/v1/multimodalembeddings",
    {
      method: "POST",
      headers: {
        Authorization: `Bearer ${process.env.VOYAGE_API_KEY}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({
        model: "voyage-multimodal-3.5",
        input_type: "document",
        inputs: inputs.map((content) => ({ content })),
      }),
    },
  );

  if (!res.ok) {
    throw new Error(`Voyage ${res.status}: ${await res.text()}`);
  }

  const json = (await res.json()) as { data: { embedding: number[] }[] };
  return json.data.map((d) => d.embedding);
}
```

Two things worth pulling out of that. `input_type: "document"` tells Voyage
this is corpus content (not a query). When you embed a user's query at
search time, switch it to `"query"`. The model treats them slightly
differently and using the right one helps relevance.

Second, `image_url` is a public URL Voyage's servers can fetch. If your
screenshots aren't public you can send `image_base64` instead and skip the
URL altogether. We'll use both in this series; the URL version is cheaper
for big libraries, the base64 version is what you reach for during
development.

## Ingest a sliver of the corpus

This is the smallest complete ingestion loop. Two text chunks and one
screenshot, all hitting the same `embed` call, all landing in the same
table.

The text comes from the [`CREATE INDEX`
page](https://www.postgresql.org/docs/current/sql-createindex.html) in the
PostgreSQL manual, specifically its "Building Indexes Concurrently"
section. The screenshot is the error-output panel from the [pgAdmin query
tool
docs](https://www.pgadmin.org/docs/pgadmin4/latest/query_tool.html). Real
sources, so you can check a retrieved chunk against the page it came from
instead of trusting a paraphrase I wrote. Note the `sourceUrl` on every
record: you want it from day one, because the moment retrieval returns
something surprising the first question is always "where did this come
from?"

```ts
// src/ingest.ts
import { db } from "./db";
import { chunks } from "./db/schema";
import { embed } from "./embed";

// postgresql.org/docs/current/sql-createindex.html, "Building Indexes
// Concurrently". Excerpted verbatim so a retrieved chunk can be checked
// against the published page.
const CREATE_INDEX_URL =
  "https://www.postgresql.org/docs/current/sql-createindex.html";

const examples = [
  {
    kind: "text" as const,
    source: "postgres-docs",
    sourceUrl: CREATE_INDEX_URL,
    body:
      "PostgreSQL will build the index without taking any locks that prevent " +
      "concurrent inserts, updates, or deletes on the table; whereas a " +
      "standard index build locks out writes (but not reads) on the table " +
      "until it's done.",
    metadata: { pageId: "sql-createindex", section: "concurrently", chunkIndex: 0 },
  },
  {
    kind: "text" as const,
    source: "postgres-docs",
    sourceUrl: CREATE_INDEX_URL,
    body:
      "If a problem arises while scanning the table, such as a deadlock or a " +
      "uniqueness violation in a unique index, the CREATE INDEX command will " +
      "fail but leave behind an \"invalid\" index. The recommended recovery " +
      "method in such cases is to drop the index and try again.",
    metadata: { pageId: "sql-createindex", section: "concurrently", chunkIndex: 1 },
  },
  {
    kind: "image" as const,
    source: "pgadmin-docs",
    sourceUrl: "https://www.pgadmin.org/docs/pgadmin4/latest/query_tool.html",
    url:
      "https://www.pgadmin.org/static/docs/pgadmin4-9.17-docs/_images/" +
      "query_output_error.png",
    metadata: { pageId: "query_tool", section: "error-output" },
  },
];

const pieces: import("./embed").VoyagePiece[][] = examples.map((ex) =>
  ex.kind === "text"
    ? [{ type: "text", text: ex.body }]
    : [{ type: "image_url", image_url: ex.url }],
);

const vectors = await embed(pieces);

await db.insert(chunks).values(
  examples.map((ex, i) => ({
    source: ex.source,
    body: ex.kind === "text" ? ex.body : null,
    embedding: vectors[i],
    metadata:
      ex.kind === "text"
        ? {
            kind: "text",
            sourceUrl: ex.sourceUrl,
            pageId: ex.metadata.pageId,
            section: ex.metadata.section,
            chunkIndex: ex.metadata.chunkIndex,
          }
        : {
            kind: "image",
            url: ex.url,
            sourceUrl: ex.sourceUrl,
            pageId: ex.metadata.pageId,
            section: ex.metadata.section,
          },
  })),
);
```

That's three rows, two modalities, one column. In Part 3 we replace the
hand-written `examples` array with a real ingestion pipeline (chunking,
batching, retries, image preprocessing) but the shape stays the same.


A note on batching while it's fresh: Voyage's multimodal endpoint accepts a
list of inputs per call. Batching is how you keep latency and cost sane.
Start with 16 inputs per call for mixed text/image, smaller if your images
are large. Wrap the call in a retry helper with exponential backoff; a 429
on a single batch shouldn't take down a backfill.

## Run the first real query

Embed the user's question with `input_type: "query"`, then ask Postgres for
the nearest rows by cosine distance:

```ts
// src/search.ts
import { cosineDistance, sql } from "drizzle-orm";
import { db } from "./db";
import { chunks } from "./db/schema";
import { embed } from "./embed";

export async function search(query: string, k = 10) {
  const [queryVector] = await embed(
    [[{ type: "text", text: query }]],
    // Tell Voyage this is a query, not a document. Set the type at the
    // call site; we'll thread it through `embed` in Part 3.
  );

  const similarity = sql<number>`1 - (${cosineDistance(
    chunks.embedding,
    queryVector,
  )})`;

  return db
    .select({
      id: chunks.id,
      source: chunks.source,
      body: chunks.body,
      metadata: chunks.metadata,
      similarity,
    })
    .from(chunks)
    .orderBy((t) => sql`${t.similarity} DESC`)
    .limit(k);
}
```

Call it with `search("recovering from a failed concurrent index build")` and
you get back a ranked mix of rows: the two paragraphs from the Postgres
docs and the screenshot of the error message, ordered by how close they sit
to the query in vector space. Same index, same query, both modalities. The
multimodal promise from Part 1 is no longer abstract.

(If you want to see the full retrieval-augmented version of this pattern
end to end in TypeScript with hybrid search and an LLM,
[here's the post that does it](https://omiid.me/notebook/53/rag-with-postgres-drizzle-and-pgvector).
We do the proper version in Part 6, but the post is good context.)

## From the trenches

A few things worth knowing now so they don't sting later.

**Don't embed during a web request.** Voyage calls take hundreds of
milliseconds, sometimes more. Ingestion belongs in a background job or a
script. Query-time embedding (one input) is fine to do inline; corpus
embedding is not.

**Your image format matters.** PNG and JPEG work everywhere. HEIC and AVIF
fail in places you wouldn't expect. Normalize on ingest (sharp,
ImageMagick, whatever you reach for) to one of the safe formats before
sending to Voyage or storing the URL.

**Pin your `pgvector` version in the image tag.** `pgvector/pgvector:pg16`
floats; that's fine for a learn-along but pin to a specific tag
(`pgvector/pgvector:0.8.1-pg16` or similar) for anything you operate. The
index file format has shifted between minor versions and an unexpected
upgrade can force a rebuild.

**`text-embedding-3-small` is not in the same space.** Just because both
models output 1536 or 1024 dims doesn't mean their vectors are comparable.
You cannot mix outputs from different models in one column. If you switch
models, you re-embed everything. Part 8 is the entire story of doing that
without downtime.

## You have a working multimodal search

You have a self-hosted Postgres with `pgvector` enabled, and a Drizzle
schema with one vector column wide enough for `voyage-multimodal-3.5` and
an HNSW index ready to grow into. The `embed()` function takes text or
images and returns vectors in the same space. The ingest script put three
rows in the table, and `search()` returns them ordered by relevance.

That's the whole loop. Everything else in this series is making it _good_,
making it _fast_, and making it survive contact with users. Part 3 starts
with the part you'll trip on first: relevance.
