pgvector HNSW on a small VPS: how much RAM?

Will a pgvector HNSW index fit in RAM on a small VPS?

A pgvector HNSW index is fast while it sits in RAM and slow as soon as it does not. On a VPS with 4 GB or 8 GB of memory, that one fact sets your query latency, and you can settle it with arithmetic before you build anything. HNSW stands for hierarchical navigable small world. The index is a graph, a query walks it, and every hop is a page read that either hits memory or hits your disk.

Published pgvector benchmarks run on machines with more RAM than your whole plan, so their latency figures do not carry over to your box. The method does. Count the bytes first, then measure your own instance.

The arithmetic: rows times dimensions times 4 bytes

A vector value holds one 4-byte float per dimension plus an 8-byte header. That is the entire storage rule, which means you can size a table that does not exist yet.

[
  {
    "label": "384 dims",
    "bytes_per_vector": 1544,
    "payload_mib": 147.2,
    "with_index_mib": 294.5
  },
  {
    "label": "768 dims",
    "bytes_per_vector": 3080,
    "payload_mib": 293.7,
    "with_index_mib": 587.5
  },
  {
    "label": "1024 dims",
    "bytes_per_vector": 4104,
    "payload_mib": 391.4,
    "with_index_mib": 782.8
  },
  {
    "label": "1536 dims",
    "bytes_per_vector": 6152,
    "payload_mib": 586.7,
    "with_index_mib": 1173.4
  },
  {
    "label": "3072 dims",
    "bytes_per_vector": 12296,
    "payload_mib": 1172.6,
    "with_index_mib": 2345.3
  }
]

The block above holds 5 common embedding widths at 100,000 rows, in MiB (mebibytes, 1,048,576 bytes each). One vector at 1536 dimensions is 6152 bytes, so 100,000 of them are 586.7 MiB. Scale it linearly for your own row count: half a million rows at that width is five times 586.7 MiB.

The with_index_mib column counts that payload twice, because pgvector's HNSW index keeps a full copy of every vector inside the index. There is no reference-only mode where the graph points back at the table. Treat the doubled number as a floor, not as an estimate. The graph links, the per-tuple headers, and the space Postgres leaves free at the end of each 8 KB page all sit on top of it.

Links are the part m controls. Each element gets up to 2 * m neighbours on the bottom layer and up to m on every layer above, and each neighbour is a pointer of a few bytes. At the default m = 16 those pointers are a small share of a 1536-dimension vector and a much larger share of a 384-dimension one. That is worth knowing before you pair a small embedding model with a large m.

Two other costs land in the same memory budget. The table also stores the chunk text the vector came from, and for short chunks that text can outweigh the vector. Queries need memory of their own, since every connection gets its own work_mem, so putting a connection pooler in front of Postgres often does more for a small box than any index setting.

Where the vectors actually live: TOAST

A heap tuple has to fit inside an 8 KB page, so Postgres pushes large values out of line into a TOAST (the oversized attribute storage technique) table. A 1536-dimension vector is over 6 KB, so it goes out of line. Two things follow. A sequential scan pays an extra fetch per row, and the planner sees the table as smaller than it really is, which skews its cost estimate for a parallel scan. pgvector's answer is to store the vectors inline:

ALTER TABLE items ALTER COLUMN embedding SET STORAGE PLAIN;

SET STORAGE applies to rows written after the change, so existing rows keep their old layout until the table is rewritten. Inline storage also means fewer rows per heap page, which costs you cache in a different place. Change it when a slow parallel scan gives you a reason.

Why page cache decides latency, not shared_buffers

Postgres reads through two caches. shared_buffers is its own, and the kernel's page cache holds everything it has read recently. pgvector's tuning note puts shared_buffers at about 25 percent of memory, which leaves the rest to the kernel, and an index page can be hot in either one. The number you are sizing against is the sum of the two.

On a 4 GB VPS that sum is smaller than it looks. The kernel, your embedding service, the application, and Postgres's per-connection memory all take their share first. When the index plus the working set of the table pushes past what is left, the kernel starts evicting pages, and the pages it evicts are the ones the next graph walk needs.

Swap changes the failure mode without fixing it, because a graph walk served from swap is still doing random reads, now with the kernel in the path as well. It is worth having so the out of memory killer does not fire, and it is not a substitute for RAM. Whether a small VPS needs swap at all deserves its own decision. If Postgres runs in a container, the container memory limit applies before the host runs out, while the process inside still reads the host's memory total and sizes itself for memory it cannot have.

The setting that decides whether the build finishes: maintenance_work_mem

pgvector builds the HNSW graph in memory. While the graph fits inside maintenance_work_mem, the build is a memory workload. When it stops fitting, the server logs this:

NOTICE:  hnsw graph no longer fits into maintenance_work_mem after 100000 tuples
DETAIL:  Building will take significantly more time.
HINT:  Increase maintenance_work_mem to speed up builds.

That notice is the difference between a build measured in minutes and one measured in hours. After that tuple, the remaining rows go in through the on-disk index one at a time, and each insert walks the graph again with random reads.

The pgvector README suggests maintenance_work_mem = '8GB', which is a figure for a machine you do not have. Set it in the session that runs the build:

SET maintenance_work_mem = '1GB';
SET max_parallel_maintenance_workers = 2;
CREATE INDEX ON items USING hnsw (embedding vector_cosine_ops) WITH (m = 16, ef_construction = 64);

Set it in postgresql.conf instead and every autovacuum worker can claim the same amount, because autovacuum falls back to maintenance_work_mem when autovacuum_work_mem is unset. Multiply that by autovacuum_max_workers and a 4 GB box has nothing left.

Watch the build from a second session:

SELECT phase, round(100.0 * blocks_done / nullif(blocks_total, 0), 1) AS "%"
FROM pg_stat_progress_create_index;

If the box runs out of memory during the build, the Linux OOM (out of memory) killer takes a postgres backend, the server restarts every connection, and the half-built index is rolled back. Watch free -m in another shell and keep maintenance_work_mem well under what it reports as available.

What m, ef_construction and ef_search cost you

Each knob buys recall with something different, so pick them against your own measurements rather than a published pair of values.

m is the number of links per element. It sets index size directly, because the neighbour lists scale with it. A higher m connects the graph better and helps recall on hard queries. On a box where the index only just fits, m is the setting most likely to push it out of cache.

ef_construction is how many candidates the build considers when it picks those links. Raising it costs build time and build memory, and it leaves the size of the finished index unchanged. Keep it at or above twice m, because the build cannot choose m good neighbours from a shorter candidate list.

ef_search is the query-time knob, and it defaults to 40:

SET hnsw.ef_search = 100;

It is the number of candidates the walk holds at once. More candidates means better recall and more pages touched. On a cached index that cost is CPU. On an index that does not fit, each extra candidate is another chance of a random read, so the setting that is cheap on a large machine is expensive on yours. Use SET LOCAL inside a transaction, so one expensive query does not leave the value raised for the rest of a pooled session.

To pick values honestly, hold a fixed set of query vectors, get exact answers by turning the index off with SET enable_indexscan = off, then compare each setting against those. Recall is the fraction of the exact top k that your index returned. Without that baseline you are tuning a number you cannot see.

Measure your own box with EXPLAIN (ANALYZE, BUFFERS)

SET track_io_timing = on;
SELECT embedding AS qv FROM items WHERE id = 1 \gset
EXPLAIN (ANALYZE, BUFFERS)
SELECT id FROM items WHERE id <> 1 ORDER BY embedding <=> :'qv' LIMIT 10;

\gset is a psql command that puts the value from a one-row result into a variable, so the test uses a real vector out of your own data instead of a placeholder.

Two things in that output matter. The node has to read Index Scan using items_embedding_idx on items, because a Seq Scan means the planner ignored the index, most often since the operator in ORDER BY does not match the operator class the index was built with. Then read the Buffers: line. shared hit came from shared_buffers. shared read went out to the kernel, which served it from page cache or from the disk underneath, and I/O Timings: tells you which of the two, because a page cache read costs almost nothing while a disk read shows up in milliseconds.

Run it twice. The second run is your best case, with everything cached. The gap between the first run and the second is the size of the problem you are about to have.

Across a real workload, watch the ratio instead of one query:

SELECT indexrelname, idx_blks_hit, idx_blks_read
FROM pg_statio_user_indexes
WHERE relname = 'items';

Then read the real sizes, rather than trusting the arithmetic at the top of this page:

SELECT pg_size_pretty(pg_relation_size('items_embedding_idx')) AS index_size,
       pg_size_pretty(pg_total_relation_size('items')) AS table_total;

Measure under the concurrency you will really run. One query at a time on an idle box is the easiest test there is, and what concurrent users do to a small server is a separate question with a separate answer.

What it looks like when the index stops fitting

The symptom looks like a broken query plan. The same query is fast, then ten times slower, then fast again, with no pattern the application can see. EXPLAIN (ANALYZE, BUFFERS) shows the same plan and the same row counts as before. The whole difference sits in shared read and I/O Timings.

The cause is the access pattern. A graph walk touches pages scattered across the index in an order that depends on the query vector, so the reads are random 8 KB reads. The sequential throughput number on a provider's plan page says nothing about that. Network-attached block storage pays a round trip per request, and the hops in a graph walk are dependent: each hop decides where the next read goes, so the reads cannot be issued in parallel and their latencies add up.

Several ordinary events push a working index over that line with nobody changing a setting. The table grows past the free memory. A backup or a reporting query reads the whole table and evicts the index pages. A restart empties both caches, so the first queries afterwards are the slow ones. A second service on the box starts claiming memory it was not claiming last month.

What to do when the index does not fit

Shrink each vector. halfvec stores 2 bytes per dimension instead of 4, which halves both the payload and the copy inside the index:

CREATE INDEX ON items USING hnsw ((embedding::halfvec(1536)) halfvec_cosine_ops);
SELECT id FROM items ORDER BY embedding::halfvec(1536) <=> :'qv' LIMIT 10;

The query has to carry the same cast or the index is not used. Half precision drops mantissa bits, so check recall against your exact baseline before you keep it. Binary quantization goes further, with binary_quantize(embedding)::bit(1536) and bit_hamming_ops, and it needs a rerank step against the original vectors to be worth using.

Use fewer dimensions. This is the biggest lever in the chart above: the same 100,000 rows cost 1172.6 MiB at 3072 dims and 147.2 MiB at 384 dims. Some models are trained so the first N dimensions stand on their own and can be truncated. Check the model card before you slice a vector, because truncating a model that was not trained for it destroys the distances. A vector column with more than 2000 dimensions cannot take an HNSW index at all, so a 3072-dimension model has to be indexed as halfvec, which allows up to 4000.

Index fewer rows. A partial index covers only the rows you search:

CREATE INDEX ON items USING hnsw (embedding vector_cosine_ops)
  WHERE archived = false;

The query has to repeat that predicate for the planner to use the index. If your data splits by tenant or by collection, one partial index per tenant keeps each graph small enough to stay resident.

There is a related trap here. With a WHERE clause and a normal HNSW index, the scan collects its ef_search candidates and the filter then discards whatever does not match, so a selective filter can return fewer rows than your LIMIT. Iterative scan fixes that by rescanning for more candidates, and it stays off until you turn it on:

SET hnsw.iterative_scan = relaxed_order;
SET hnsw.max_scan_tuples = 40000;

relaxed_order allows results slightly out of distance order and is cheaper than strict_order. hnsw.max_scan_tuples caps the extra work and defaults to 20000. Rescanning means more page reads, so on a box that is already short of cache, a partial index is usually the better answer.

Consider IVFFlat. It splits the vectors into lists and searches the lists closest to the query. It builds faster and uses less memory than HNSW, and its speed-recall trade is worse. It also has to be built after the data is loaded, because the lists come from clustering the rows that exist at build time, and recall drifts as new rows land outside those clusters. A reasonable start is lists = rows / 1000 for up to a million rows, with ivfflat.probes raised from its default of 1 until recall is acceptable. On a small VPS, a weaker index that stays in cache can beat a better index that does not.

Move the workload. When the vectors dominate a Postgres instance that also serves your application data, running a dedicated vector database on its own VPS separates the two memory budgets. When the box is simply too small, the decision turns into a cost question, and managed Postgres against a self-hosted instance is where to work it out. If this index is one stage of a larger system, size it with the rest of the stack, since a self-hosted retrieval augmented generation pipeline puts an embedding model on the same machine, competing for the same RAM.

FAQ

How much RAM does a pgvector HNSW index need?

Start from rows times dimensions times 4 bytes, double it because HNSW keeps a copy of every vector inside the index, then add the neighbour lists that m controls. For 100,000 rows at 1536 dimensions that floor is 1173.4 MiB before any overhead. The index does not have to fit in memory for queries to return correct results. It has to fit for them to be fast, because a graph walk that misses cache becomes a chain of random reads.

Why did my HNSW index build take hours?

Look in the Postgres log for hnsw graph no longer fits into maintenance_work_mem after N tuples. Past that tuple, rows are added through the on-disk index one at a time, with random reads for each one. Raise maintenance_work_mem in the session that runs CREATE INDEX, keep the value under what free -m reports as available, and follow the build with pg_stat_progress_create_index.

Why does adding a WHERE clause return fewer rows than my LIMIT?

An HNSW scan collects ef_search candidates first, and the filter then removes the ones that do not match, so a selective filter can leave you short of the row count you asked for. Raise hnsw.ef_search, set hnsw.iterative_scan so the index rescans for more candidates, or build a partial index carrying the same predicate in its own WHERE clause.

Should I use IVFFlat instead of HNSW on a small VPS?

IVFFlat uses less memory and builds faster, and it gives a worse speed-recall trade at the same accuracy target. Test it when HNSW does not fit in cache, because a smaller index that stays resident can beat a larger one that is read from disk. Build it after the data is loaded, since the lists come from clustering the rows that exist at build time.