Running a vector database on your VPS

What a vector database on a VPS actually costs you

Running a vector database on a VPS (virtual private server) deletes the problem every managed vendor is selling a fix for. Your application and your index sit on the same machine, so a search request crosses a loopback socket instead of a network. What remains is the cost that was always the real one: turning text into vectors. Behind it sit two more, the time to build the index and the RAM the index holds for as long as it serves.

That changes which decisions matter. Region and endpoint round trip stop being your concern. The product of vector count, dimensions and four bytes becomes your concern, because it decides whether the index fits in the memory you rent every month.

Where the milliseconds actually go on one box

Follow one similarity query through a self-hosted stack.

1. The query text is turned into a vector by an embedding model. On CPU that is tens to hundreds of milliseconds for a short string. On a GPU it is single digits.
2. The vector is sent to the store. Over loopback TCP or a Unix domain socket this is a fraction of a millisecond.
3. The store walks its index and returns the nearest rows.
4. Your code reads the matching text and assembles a prompt.

Step 1 is usually the largest number in that list. Step 2 is the step hosted vendors compete on, and on one box it barely exists. Do not guess the split. Time both ends on your own server.

curl http://127.0.0.1:11434/api/embed -s -o /dev/null \
  -w 'embed: %{time_total}s\n' \
  -d '{"model": "nomic-embed-text", "input": "how do I rotate the api key"}'

Then run \timing on in psql before the search query. If the first command prints embed: 0.184312s and psql answers Time: 4.201 ms, then tuning the index is the wrong job: your latency is the embedding model. Running the embedding model locally with Ollama puts step 1 on the same CPU as steps 2 and 3, so both halves compete for the same cores and the same RAM. The ingest and retrieval loop wrapped around this store is covered in the self-hosted RAG pipeline guide. RAG is retrieval augmented generation: you search your own documents and paste the best matches into a prompt.

Under about a hundred thousand vectors, scan all of them

An exhaustive scan compares the query against every stored vector. Recall is perfect by definition. It needs no index and no build step, and it cannot drift out of date behind your data.

The arithmetic tells you when it stops being fine. A scan reads n * d * 4 bytes per query, where n is the vector count and d is the dimension. At 100,000 vectors of 768 dimensions that is 307 MB per query, which a modern CPU streams in a few tens of milliseconds. At 5 million vectors it is 15 GB per query, which is not a query any more.

So store the vectors in SQLite and do the comparison in NumPy.

import sqlite3, numpy as np

db = sqlite3.connect("docs.db")
db.execute("CREATE TABLE IF NOT EXISTS docs (id INTEGER PRIMARY KEY, body TEXT, vec BLOB)")

def add(body, vec):
    v = np.asarray(vec, dtype=np.float32)
    v /= np.linalg.norm(v)
    db.execute("INSERT INTO docs (body, vec) VALUES (?, ?)", (body, v.tobytes()))
    db.commit()

def search(query_vec, k=5):
    rows = db.execute("SELECT id, body, vec FROM docs").fetchall()
    mat = np.frombuffer(b"".join(r[2] for r in rows), dtype=np.float32).reshape(len(rows), -1)
    q = np.asarray(query_vec, dtype=np.float32)
    q /= np.linalg.norm(q)
    scores = mat @ q
    return [(rows[i][0], rows[i][1], float(scores[i])) for i in np.argsort(-scores)[:k]]

Both sides are scaled to unit length, so the dot product is cosine similarity and a higher score is a closer match. Load mat once at startup instead of once per query and the SQLite read leaves the hot path entirely.

Time it on your own box before you reject it.

import time
t = time.perf_counter()
search(q)
print(f"{(time.perf_counter() - t) * 1000:.1f} ms")

The honest limits: this is one process holding the whole matrix in RAM, and it gives you no metadata filtering and no concurrent writer story. When one of those is the reason you are unhappy, move. SQLite itself is a serious server-side store, which the SQLite in production guide goes through, and if your real workload is scanning columns rather than serving rows then the DuckDB and SQLite comparison is the more useful read.

pgvector, when you already run Postgres

If your application already has a Postgres database, pgvector adds the least new surface. It is an extension, not a service. Vectors live in a normal table next to the row they describe, so a filtered search is a WHERE clause instead of a second system to keep in sync.

Ubuntu 24.04 carries it in the universe component.

sudo apt update
sudo apt install -y postgresql-16-pgvector
sudo -u postgres psql -d yourdb -c 'CREATE EXTENSION vector;'

That package is pgvector 0.6.0 as of August 2026, which is well behind upstream. Iterative index scans in particular need 0.8, so take those from the PostgreSQL project's own repository.

sudo apt install -y postgresql-common
sudo /usr/share/postgresql-common/pgdg/apt.postgresql.org.sh
sudo apt install -y postgresql-17-pgvector

Replace 17 with your server's major version, which sudo -u postgres psql -tAc 'SHOW server_version' prints. Install the extension package built for the wrong major version and CREATE EXTENSION fails, because Postgres only looks in the share directory of the version that is running:

ERROR:  could not open extension control file "/usr/share/postgresql/16/extension/vector.control": No such file or directory

The schema is ordinary SQL with one new type.

CREATE TABLE chunks (
  id        bigserial PRIMARY KEY,
  doc_id    bigint NOT NULL,
  body      text   NOT NULL,
  embedding vector(768)
);

SELECT id, body FROM chunks
ORDER BY embedding <=> '[0.013, -0.021, 0.004]'
LIMIT 5;

<=> is cosine distance, <-> is L2 (Euclidean) distance, and <#> is negative inner product. Use the one your embedding model was trained for. Pick the wrong one and nothing errors, your results are just quietly worse.

With no index that query is an exact search over every row, which is the Postgres version of the brute force above and has the same perfect recall. Raising max_parallel_workers_per_gather puts more cores on it. Do this first and index later, because now you have a recall baseline to measure the index against.

Qdrant, when the index outgrows the database

Qdrant is a dedicated vector store written in Rust. It earns its place when the index is large enough that you would rather its build did not compete with your application's Postgres, or when you want payload filtering and quantisation that pgvector does not offer.

docker run -d --name qdrant \
  -p 127.0.0.1:6333:6333 -p 127.0.0.1:6334:6334 \
  -e QDRANT__SERVICE__API_KEY="$(openssl rand -hex 32)" \
  -v "$(pwd)/qdrant_storage:/qdrant/storage:z" \
  qdrant/qdrant

Port 6333 serves the REST (representational state transfer) API and a dashboard at /dashboard, and 6334 serves gRPC. Two details matter on a public VPS. Qdrant's own documentation says the service runs by default "with no encryption or authentication", and the -p 6333:6333 from the quickstart binds to every interface, which Docker publishes past a ufw rule because it writes its own forwarding rules. Bind to 127.0.0.1 and set an API key. A Qdrant instance reachable on a public IP with no key is a public copy of your documents.

curl -s http://127.0.0.1:6333/collections -H "api-key: $QDRANT_API_KEY"

A healthy answer looks like {"result":{"collections":[]},"status":"ok","time":0.00002}. Getting {"status":{"error":"Unauthorized"}} back means the header name or the key is wrong, and getting nothing at all means the container is not running or is bound somewhere else. Whether that container is the right shape for your box is the ordinary stateful-service question, so the Docker versus host database comparison applies here unchanged.

Chroma, and what it is for

Chroma is the shortest path from nothing to a working retrieval demo.

pip install chromadb
chroma run --path /srv/chroma

That serves on port 8000, and chromadb.HttpClient(host="localhost", port=8000) connects to it. Chroma ships a default embedding function, so a first prototype needs no separate model server at all.

Be honest about the trade. Chroma is pleasant because it hides the decisions this guide is about: which distance metric, and how much RAM the result will hold. That is correct for a prototype and wrong for the thing you get paged about. If your data already lives in Postgres, moving it to Chroma adds a process and a synchronisation problem in order to solve a problem pgvector does not have.

How much RAM will the index need

Start from the raw vectors, because they are the floor and no tuning moves them.

bytes = number_of_vectors * dimensions * 4

Four bytes is one 32-bit float per dimension. Qdrant's capacity planning documentation adds a 1.5 multiplier for metadata and the temporary segments made during optimisation:

memory_size = number_of_vectors * vector_dimension * 4 bytes * 1.5

Here is that formula run over one million vectors, at the dimensions real embedding models produce.

[
  {
    "label": "384 dims",
    "raw_gib": 1.43,
    "with_overhead_gib": 2.15
  },
  {
    "label": "768 dims",
    "raw_gib": 2.86,
    "with_overhead_gib": 4.29
  },
  {
    "label": "1024 dims",
    "raw_gib": 3.81,
    "with_overhead_gib": 5.72
  },
  {
    "label": "1536 dims",
    "raw_gib": 5.72,
    "with_overhead_gib": 8.58
  },
  {
    "label": "3072 dims",
    "raw_gib": 11.44,
    "with_overhead_gib": 17.17
  }
]

Those are the formula's output in gibibytes (GiB), not a measurement. Read them as the size of the hole you have to leave in memory. A 768-dimension model such as nomic-embed-text over a million chunks wants about 4.29 GiB, which fits an 8 GB plan with room left for Postgres. The same corpus at 3072 dimensions wants 17.17 GiB and does not.

The lever is the first column of that chart, not the last. Halving the dimension halves every byte downstream of it, forever. A 768-dimension model that scores a little worse on a public leaderboard is often the better engineering choice on a VPS. pgvector's halfvec type then stores 16-bit floats, which halves the bytes again, and it indexes through an expression:

CREATE INDEX ON chunks USING hnsw ((embedding::halfvec(768)) halfvec_cosine_ops);

One limit to know before you pick a model. pgvector's vector type accepts up to 16,000 dimensions, but its HNSW and IVFFlat indexes only cover 2,000. Above that you index a halfvec cast, which reaches 4,000, or you do not index at all.

What m and ef_construction cost at build time

HNSW (hierarchical navigable small world) is the index pgvector and Qdrant both reach for. It is a layered graph. Every vector is a node with links to nearby nodes, and a search hops along those links toward the query instead of reading everything.

m is how many links each node keeps. The Faiss documentation gives HNSW memory as (d * 4 + m * 2 * 4) bytes per vector and recommends keeping m between 4 and 64. Run that at 768 dimensions over a million vectors.

[
  {
    "label": "m = 8",
    "link_bytes_per_vector": 64,
    "total_gib": 2.92
  },
  {
    "label": "m = 16 (default)",
    "link_bytes_per_vector": 128,
    "total_gib": 2.98
  },
  {
    "label": "m = 32",
    "link_bytes_per_vector": 256,
    "total_gib": 3.1
  },
  {
    "label": "m = 64",
    "link_bytes_per_vector": 512,
    "total_gib": 3.34
  }
]

The useful surprise is the size of that difference. Going from the default m = 16 to m = 64 adds 512 bytes of links per vector against 3072 bytes of vector data, so the total moves from 2.98 GiB to 3.34 GiB. That is about 12 percent. At these dimensions m is not where your memory goes. The vectors are.

What m really costs is build time and insert time, because placing a node means finding and linking that many neighbours. ef_construction is the size of the candidate list the builder considers while it places each node. Raising it produces a better graph and a slower build, and it does not change the finished index size at all.

SET maintenance_work_mem = '4GB';
SET max_parallel_maintenance_workers = 7;
CREATE INDEX ON chunks USING hnsw (embedding vector_cosine_ops)
  WITH (m = 16, ef_construction = 64);

maintenance_work_mem is the setting that decides whether the build takes minutes or hours, because pgvector assembles the graph in memory when it fits. When it stops fitting, pgvector says so:

NOTICE:  hnsw graph no longer fits into maintenance_work_mem after 100000 tuples
DETAIL:  Building will take significantly more time.
HINT:  Increase maintenance_work_mem to speed up builds.

That notice is the most useful line pgvector prints. It means the build has dropped onto a much slower path, so cancel it, raise the setting past the RAM figure you calculated above, and start again. Watch progress from a second session:

SELECT phase, round(100.0 * blocks_done / nullif(blocks_total, 0), 1) AS pct
FROM pg_stat_progress_create_index;

HNSW reports initializing and then loading tuples. A build sitting at a low percentage for a long time on a disk that is not busy is the maintenance_work_mem problem, not a stuck query.

Two facts worth planning around. pgvector's README says HNSW "has slower build times and uses more memory" than IVFFlat, and buys a better speed against recall trade in return. And HNSW can be created on an empty table, while IVFFlat has to run k-means over representative data first, so building it on an empty table gives poor recall. On a fresh schema HNSW is the one you can create up front.

ef_search: the knob you tune after the build

m and ef_construction are frozen into the index. ef_search is not. It sets how many candidates the search keeps while it walks the graph, and you change it per session or per query.

SET hnsw.ef_search = 100;
SELECT id, body FROM chunks ORDER BY embedding <=> $1 LIMIT 5;

The default is 40. Raise it and recall climbs, and latency climbs with it. Lower it and both fall. This is the only recall control you can move without a rebuild, so tune it against a fixed set of queries whose correct answers you already know, and stop when recall stops improving.

One trap. ef_search interacts badly with a selective WHERE clause, because the index hands back a fixed number of candidates and the filter is applied afterwards. A filter that rejects most rows can leave you with fewer than LIMIT results while matching rows sit in the table. pgvector 0.8 answers this with iterative scans:

SET hnsw.iterative_scan = relaxed_order;

The index is then re-scanned for more candidates until the limit is satisfied, up to hnsw.max_scan_tuples, which defaults to 20000. strict_order keeps exact distance ordering and costs more. This is the feature the Ubuntu 0.6.0 package does not have, and missing rows under a filter is how you find out.

Why the index has to fit in RAM

An HNSW search is a walk over a graph. Each hop reads a node stored somewhere unrelated to the previous one, so the access pattern is close to random and read-ahead does not help. While the graph is in RAM every hop is a memory reference. Once it is not, a hop can become a disk read, and a search that touches a few hundred nodes becomes a few hundred reads.

Qdrant's documentation puts a shape on it: "if you store half as many vectors in RAM, search latency will roughly double." Plan against that sentence.

When the index genuinely will not fit, every option is a trade you should make on purpose.

• Memory-map the vectors so the operating system caches hot pages and leaves cold ones on disk. This needs fast NVMe (non-volatile memory express) storage underneath to be tolerable.
• Quantise, storing each dimension as one byte instead of four. That cuts vector bytes by four for a small, measurable recall cost.
• Cast to halfvec in pgvector, which halves the bytes with less recall loss than one-byte quantisation.
• Embed with a smaller model. This is the cheapest fix and the one people skip, because it means re-embedding the corpus.

Failure modes and the strings you will see

could not open extension control file. The pgvector package for the running Postgres major version is not installed. Print the version with sudo -u postgres psql -tAc 'SHOW server_version' and install the matching postgresql-NN-pgvector.

ERROR: expected 768 dimensions, not 1536. The column type and the model disagree. You changed embedding models and did not re-embed. There is no partial fix here, because vectors from two different models are not comparable at all, so every row has to be regenerated.

The query is slow and EXPLAIN shows a sequential scan. The index operator class and the query operator do not match. vector_cosine_ops only serves <=>. Run EXPLAIN ANALYZE on the query and look for Index Scan using ... on chunks. If you see Seq Scan on chunks instead, rebuild the index with the operator class that matches the operator you actually query with.

Fewer rows than LIMIT, and a WHERE clause is present. That is the filtering trap above. Raise hnsw.ef_search, or move to pgvector 0.8 and set hnsw.iterative_scan.

The index build ends with the process gone and no error in psql. maintenance_work_mem set to most of the machine, while shared_buffers and your application also want memory, ends at the kernel out-of-memory killer. sudo dmesg -T | grep -i 'killed process' shows the line naming postgres. Lower the setting, or build the index on a bigger plan and restore the dump.

Picking one

If you already run Postgres and you have fewer than a few million vectors, use pgvector. The index sits beside the data, filtering is a WHERE clause, and your existing backup already covers it. If the index is large enough to want its own memory ceiling, or you need heavy payload filtering, run Qdrant beside it and accept a second service to operate.

Below roughly a hundred thousand vectors, measure the brute-force scan before you install anything. An exhaustive search with perfect recall and no build step is not a compromise at that size. It is the correct answer, and reaching for an approximate index instead means taking on tuning and RAM pressure in exchange for milliseconds you were not spending.

FAQ

Do I need a dedicated vector database, or is Postgres enough?

If your data already lives in Postgres, pgvector is enough far longer than most comparisons suggest. It stores vectors in an ordinary column, so a filtered search is a WHERE clause and your existing backup covers the index. Move to a dedicated store like Qdrant when the vector workload needs its own memory ceiling, or when you need payload filtering and quantisation that pgvector does not provide.

How many vectors can one VPS hold?

Work it out rather than guessing, using number_of_vectors * dimensions * 4 bytes * 1.5. One million 768-dimension vectors comes to roughly 4.3 GiB, so an 8 GB plan holds that with room for Postgres. One million 3072-dimension vectors comes to roughly 17 GiB and needs a much larger plan. The number that moves this most is the dimension of your embedding model, so choose that model with the memory bill in view.

Why is my vector search slow when the index is on the same machine?

The network is not your problem on one box, so look at the two things that are. First, time the embedding call on its own, because generating the query vector on CPU often takes far longer than the search itself. Second, check the index is in RAM. An HNSW search hops randomly through a graph, so once the graph spills to disk each hop can become a disk read, and Qdrant's own guidance is that halving the vectors held in RAM roughly doubles search latency.

Should I build an HNSW index at all?

Not below roughly a hundred thousand vectors. An exhaustive scan reads n * d * 4 bytes per query, which is 307 MB at 100,000 vectors of 768 dimensions, and a modern CPU streams that in tens of milliseconds with perfect recall and no build step. Measure the scan on your own hardware first. Build the index when the measured scan time is genuinely too slow, not because a benchmark article said so.

What does raising m actually cost me?

Build time and insert time, far more than memory. At 768 dimensions, going from the default m = 16 to m = 64 adds 512 bytes of graph links per vector against 3072 bytes of vector data, so total memory rises by about 12 percent. Every insert, though, has to find and link four times as many neighbours. Tune ef_search first, since it costs nothing to change and needs no rebuild.