Vector database for VPS: pgvector, Qdrant or Chroma?

Wetin vector database for VPS really dey cost you

Running vector database for VPS (virtual private server) dey remove the problem wey every managed vendor dey sell solution for. Your application and index dey for the same machine, so search request dey pass through loopback socket instead of network. Wetin remain na the cost wey always be the real one: turning text into vectors. Two more costs dey behind am: the time to build the index and the RAM wey the index dey occupy as long as e dey serve requests.

This one change which decisions matter. Region and endpoint round trip no longer be your concern. The product of vector count, dimensions, and four bytes na your concern, because e determine whether the index go fit inside the memory wey you rent every month.

Where milliseconds dey really go for one box

Follow one similarity query through a self-hosted stack.

1. An embedding model dey turn the query text into vector. For CPU, this fit take tens to hundreds of milliseconds for short string. For GPU, e fit take single digits.
2. The vector dey go the store. Over loopback TCP or Unix domain socket, this dey take fraction of one millisecond.
3. The store dey scan its index and return the nearest rows.
4. Your code dey read the matching text and assemble prompt.

Step 1 usually get the biggest number for that list. Step 2 na the step wey hosted vendors dey compete on, and for one box e almost no dey exist. No guess the split. Time both ends for your own server.

curl http://127.0.0.1:11434/api/embed -s -o /dev/null \
  -w 'embed: %{time_total}s\n' \
  -d '{"model": "nomic-embed-text", "input": "how do I rotate the api key"}'

Then run \timing on inside psql before the search query. If the first command prints embed: 0.184312s and psql answers Time: 4.201 ms, index tuning na the wrong job: na the embedding model dey cause your latency. Run the embedding model locally with Ollama puts step 1 for the same CPU wey steps 2 and 3 dey use, so both halves dey compete for the same cores and same RAM. The ingest and retrieval loop wey dey around this store dey covered for the self-hosted RAG pipeline guide. RAG mean retrieval augmented generation: you search your own documents and paste the best matches inside prompt.

Under about a hundred thousand vectors, scan all of dem

Exhaustive scan dey compare query against every vector wey you store. Recall dey perfect by definition. E no need index or build step, and e no fit become out of date compared with your data.

The arithmetic go tell you when e stop make sense. Scan dey read n * d * 4 bytes for each query, where n na vector count and d na dimension. For 100,000 vectors with 768 dimensions, na 307 MB for each query. Modern CPU fit stream am within few tens of milliseconds. For 5 million vectors, na 15 GB for each query. At that point, e no really be query again.

So store the vectors for SQLite and do the comparison with NumPy.

import sqlite3, numpy as np

db = sqlite3.connect("docs.db")
db.execute("CREATE TABLE IF NOT EXISTS docs (id INTEGER PRIMARY KEY, body TEXT, vec BLOB)")

def add(body, vec):
    v = np.asarray(vec, dtype=np.float32)
    v /= np.linalg.norm(v)
    db.execute("INSERT INTO docs (body, vec) VALUES (?, ?)", (body, v.tobytes()))
    db.commit()

def search(query_vec, k=5):
    rows = db.execute("SELECT id, body, vec FROM docs").fetchall()
    mat = np.frombuffer(b"".join(r[2] for r in rows), dtype=np.float32).reshape(len(rows), -1)
    q = np.asarray(query_vec, dtype=np.float32)
    q /= np.linalg.norm(q)
    scores = mat @ q
    return [(rows[i][0], rows[i][1], float(scores[i])) for i in np.argsort(-scores)[:k]]

Both sides dey scale to unit length, so dot product na cosine similarity, and higher score mean say the match dey closer. Load mat once when the process start instead of once for each query. This remove SQLite read completely from the hot path.

Measure the time for your own machine before you reject this approach.

import time
t = time.perf_counter()
search(q)
print(f"{(time.perf_counter() - t) * 1000:.1f} ms")

The honest limits be these: na one process dey hold the complete matrix for RAM, and e no give you metadata filtering or proper support for concurrent writers. When any of these na why you no happy, move go another option. SQLite itself na serious server-side store, as this guide about SQLite for production explain, and if your real workload dey scan columns instead of serving rows, then this comparison between DuckDB and SQLite go be the more useful read.

pgvector, if you already dey run Postgres

If your application already get Postgres database, pgvector na the option wey add the least new surface. E be extension, no be service. Vectors dey inside normal table beside the row wey dem describe, so filtered search na a WHERE clause instead of another system wey you need keep in sync.

Ubuntu 24.04 get am for the universe component.

sudo apt update
sudo apt install -y postgresql-16-pgvector
sudo -u postgres psql -d yourdb -c 'CREATE EXTENSION vector;'

That package na pgvector 0.6.0 as of October 2026, and e dey far behind upstream. Iterative index scans especially need 0.8, so get those ones from PostgreSQL project's own repository.

sudo apt install -y postgresql-common
sudo /usr/share/postgresql-common/pgdg/apt.postgresql.org.sh
sudo apt install -y postgresql-17-pgvector

Replace 17 with your server's major version, wey sudo -u postgres psql -tAc 'SHOW server_version' go print. If you install extension package wey build for wrong major version, CREATE EXTENSION go fail, because Postgres only dey look inside the share directory for the version wey dey run:

ERROR:  could not open extension control file "/usr/share/postgresql/16/extension/vector.control": No such file or directory

The schema na ordinary SQL with one new type.

CREATE TABLE chunks (
  id        bigserial PRIMARY KEY,
  doc_id    bigint NOT NULL,
  body      text   NOT NULL,
  embedding vector(768)
);

SELECT id, body FROM chunks
ORDER BY embedding <=> '[0.013, -0.021, 0.004]'
LIMIT 5;

<=> na cosine distance, <-> na L2 (Euclidean) distance, and <#> na negative inner product. Use the one wey your embedding model train for. If you pick wrong one, nothing go show error; na only your results go quietly worse.

If no index dey, that query na exact search across every row. Na the Postgres version of the brute force wey we mention before, and e get the same perfect recall. Increasing max_parallel_workers_per_gather go make more cores work on am. Do this first, then add index later, because now you get recall baseline wey you fit use measure the index.

Qdrant, wen index outgrow database

Qdrant na dedicated vector store wey dem write for Rust. E make sense when index don big reach the point wey you no want make e build dey compete with your application Postgres, or when you want payload filtering plus built-in scalar or product quantisation, since pgvector documentation only support half precision and binary quantisation.

docker run -d --name qdrant \
  -p 127.0.0.1:6333:6333 -p 127.0.0.1:6334:6334 \
  -e QDRANT__SERVICE__API_KEY="$(openssl rand -hex 32)" \
  -v "$(pwd)/qdrant_storage:/qdrant/storage:z" \
  qdrant/qdrant

Port 6333 dey serve REST (representational state transfer) API and dashboard for /dashboard, while 6334 dey serve gRPC. Two things important for public VPS. Qdrant own documentation talk say service dey run by default "with no encryption or authentication", and the -p 6333:6333 from quickstart bind to every interface. Docker publish am past ufw rule because e dey write im own forwarding rules. Bind to 127.0.0.1 and set API key. Qdrant instance wey public IP fit reach without key na public copy of your documents.

curl -s http://127.0.0.1:6333/collections -H "api-key: $QDRANT_API_KEY"

Healthy response suppose look like {"result":{"collections":[]},"status":"ok","time":0.00002}. If you get {"status":{"error":"Unauthorized"}} back, e mean say header name or key dey wrong. If you no get anything at all, e mean say container no dey run or e bind somewhere else. Whether that container get the correct setup for your server na the normal stateful-service question, so the Docker versus host database comparison still apply here.

Chroma, and wetin e dey for

Chroma na the shortest way from nothing reach working retrieval demo.

pip install chromadb
chroma run --path /srv/chroma

That one dey listen for port 8000, and chromadb.HttpClient(host="localhost", port=8000) dey connect to am. Chroma get default embedding function, so first prototype no need separate model server at all.

Make we talk true about the trade-off. Chroma dey easy because e hide the decisions wey this guide dey discuss: which distance metric to use, and how much RAM the result go occupy. That one correct for prototype, but e wrong for the thing wey dem go page you about. If your data already dey Postgres, moving am to Chroma go add one process and one synchronisation problem just to solve problem wey pgvector no get.

How much RAM index go need

Start with the raw vectors, because na dem be the floor and no tuning fit move dem.

bytes = number_of_vectors * dimensions * 4

Four bytes na one 32-bit float for each dimension. Qdrant guide for collections wey don outgrow RAM add 1.5 multiplier for metadata, indexes, and temporary segments wey optimisation dey create:

memory_size = number_of_vectors * vector_dimension * 4 bytes * 1.5

Na this formula dem run over one million vectors, for the dimensions wey real embedding models dey produce.

[
  {
    "label": "384 dims",
    "raw_gib": 1.43,
    "with_overhead_gib": 2.15
  },
  {
    "label": "768 dims",
    "raw_gib": 2.86,
    "with_overhead_gib": 4.29
  },
  {
    "label": "1024 dims",
    "raw_gib": 3.81,
    "with_overhead_gib": 5.72
  },
  {
    "label": "1536 dims",
    "raw_gib": 5.72,
    "with_overhead_gib": 8.58
  },
  {
    "label": "3072 dims",
    "raw_gib": 11.44,
    "with_overhead_gib": 17.17
  }
]

Na the formula output for gibibytes (GiB), no be measurement. Read dem as the size of the space wey you need leave for memory. A 768-dimension model like nomic-embed-text over one million chunks need about 4.29 GiB. E fit inside 8 GB plan with space still remain for Postgres. The same corpus for 3072 dimensions need 17.17 GiB and e no fit.

The main lever na the first column of that chart, no be the last one. If you halve the dimension, you halve every byte wey come after am, forever. A 768-dimension model wey score small lower for public leaderboard often na the better engineering choice for VPS. pgvector's halfvec type then store 16-bit floats, wey halve the bytes again, and e index through an expression:

CREATE INDEX ON chunks USING hnsw ((embedding::halfvec(768)) halfvec_cosine_ops);

Know this limit before you choose model. pgvector's vector type accept up to 16,000 dimensions, but e be only HNSW and IVFFlat indexes cover up to 2,000. A halfvec cast reach 4,000. Above that, pgvector documents binary quantisation: index a binary_quantize(embedding)::bit(N) expression with bit_hamming_ops, wey cover up to 64,000 dimensions, then re-rank the top candidates against the original vectors.

Wetin m and ef_construction cost for build time

HNSW (hierarchical navigable small world) na the index wey pgvector and Qdrant both dey use. Na layered graph be this. Every vector na one node wey get links to nearby nodes. Search dey move through those links go query instead of reading everything.

m na how many links each node dey keep. Faiss documentation give HNSW memory as (d * 4 + m * 2 * 4) bytes per vector and recommend say make m remain between 4 and 64. Run that for 768 dimensions across one million vectors.

[
  {
    "label": "m = 8",
    "link_bytes_per_vector": 64,
    "total_gib": 2.92
  },
  {
    "label": "m = 16 (default)",
    "link_bytes_per_vector": 128,
    "total_gib": 2.98
  },
  {
    "label": "m = 32",
    "link_bytes_per_vector": 256,
    "total_gib": 3.1
  },
  {
    "label": "m = 64",
    "link_bytes_per_vector": 512,
    "total_gib": 3.34
  }
]

The useful surprise na how big that difference be. If you move from default m = 16 go m = 64, the links go grow from 128 to 512 bytes per vector. This na extra 384 bytes against 3072 bytes of vector data, so total go move from 2.98 GiB go 3.34 GiB. That na about 12 percent. For these dimensions, m no be where your memory dey go. Na the vectors.

Wetin m really cost na build time and insert time, because to place one node, system must find and link that many neighbours. ef_construction na the size of the candidate list wey builder dey check while e dey place each node. If you raise am, graph go better but build go slower. E no change the size of the finished index at all.

SET maintenance_work_mem = '4GB';
SET max_parallel_maintenance_workers = 7;
CREATE INDEX ON chunks USING hnsw (embedding vector_cosine_ops)
  WITH (m = 16, ef_construction = 64);

maintenance_work_mem na the setting wey decide whether build go take minutes or hours, because pgvector dey assemble the graph for memory when e fit. When e no fit again, pgvector go show this:

NOTICE:  hnsw graph no longer fits into maintenance_work_mem after 100000 tuples
DETAIL:  Building will take significantly more time.
HINT:  Increase maintenance_work_mem to speed up builds.

That notice na the most useful line wey pgvector dey print. E mean say build don move to a much slower path. Cancel am, raise the setting pass the RAM figure wey you calculate above, then start again. Monitor progress from another session:

SELECT phase, round(100.0 * blocks_done / nullif(blocks_total, 0), 1) AS pct
FROM pg_stat_progress_create_index;

HNSW dey report initializing and then loading tuples. If build stay for low percentage for long time while disk no dey busy, na maintenance_work_mem problem be that, no be query wey hang.

Two facts dey important when you dey plan. pgvector README talk say HNSW "has slower build times and uses more memory" than IVFFlat, but e give better speed-versus-recall trade-off. HNSW fit also be created for empty table, but IVFFlat must first run k-means over representative data. So if you build IVFFlat for empty table, recall go poor. For fresh schema, HNSW na the one wey you fit create up front.

ef_search: na knob wey you tune after the build

m and ef_construction don freeze inside the index. ef_search no freeze. E set how many candidates the search go keep while e dey walk through the graph, and you fit change am per session or per query.

SET hnsw.ef_search = 100;
SELECT id, body FROM chunks ORDER BY embedding <=> $1 LIMIT 5;

The default na 40. Raise am, recall go increase, and latency go increase too. Lower am, both go reduce. Na this only recall control you fit change without rebuild, so tune am against fixed set of queries wey you already know their correct answers, then stop when recall stop improving.

One trap dey. ef_search no work well with selective WHERE clause, because the index dey return fixed number of candidates and dem apply the filter afterwards. Filter wey reject most rows fit leave you with fewer than LIMIT results, even when matching rows still dey inside the table. pgvector 0.8 solve this with iterative scans:

SET hnsw.iterative_scan = relaxed_order;

The index go scan again for more candidates until e satisfy the limit, up to hnsw.max_scan_tuples, wey default na 20000. strict_order maintain exact distance ordering, but e cost more. Na this feature the Ubuntu 0.6.0 package no get, and missing rows under a filter na how you go notice am.

Why index gree fit inside RAM

HNSW search na walk over graph. Every hop dey read node wey dey somewhere different from the previous one, so the access pattern almost random, and read-ahead no help. As long as graph dey inside RAM, every hop na memory reference. Once e no dey there again, one hop fit become disk read, and search wey touch few hundred nodes fit cause few hundred reads.

Qdrant documentation explain am like this: "if you store half as many vectors in RAM, search latency will roughly double." Plan with this sentence for mind.

When index really no go fit, every option na trade-off wey you suppose choose deliberately.

• Memory-map the vectors so operating system fit cache hot pages and leave cold ones for disk. You need fast NVMe (non-volatile memory express) storage underneath before this go tolerable.
• Quantise, by storing each dimension as one byte instead of four. This reduce vector bytes by four, with small but measurable recall cost.
• Cast to halfvec for pgvector, wey cut the bytes by half and cause less recall loss than one-byte quantisation.
• Embed with smaller model. This na the cheapest fix and the one people dey skip, because e mean say you must re-embed the corpus.

Wahala wey fit happen and strings wey you go see

could not open extension control file. pgvector package for the Postgres major version wey dey run no dey installed. Print the version with sudo -u postgres psql -tAc 'SHOW server_version' and install the matching postgresql-NN-pgvector.

ERROR: expected 768 dimensions, not 1536. Column type and model no agree. You change embedding models but you no re-embed. No partial fix dey here, because vectors from two different models no comparable at all, so you must regenerate every row.

Query dey slow and EXPLAIN show sequential scan. Index operator class and query operator no match. vector_cosine_ops only support <=>. Run EXPLAIN ANALYZE for the query and look for Index Scan using ... on chunks. If you see Seq Scan on chunks instead, rebuild the index with the operator class wey match the operator wey you actually dey query with.

Rows dey fewer than LIMIT, and WHERE clause dey present. Na the filtering trap wey we talk about above. Increase hnsw.ef_search, or move to pgvector 0.8 and set hnsw.iterative_scan.

Index build end with the process gone and no error for psql. If you set maintenance_work_mem to most of the machine, while shared_buffers and your application still need memory, kernel out-of-memory killer fit terminate the process. sudo dmesg -T | grep -i 'killed process' go show the line wey name postgres. Reduce the setting, or build the index on a bigger plan and restore the dump.

Wey one to pick

If you already dey run Postgres and you get less than a few million vectors, use pgvector. The index dey beside the data, filtering na a WHERE clause, and your existing backup already cover am. If the index big reach where e need im own memory ceiling, or you need heavy payload filtering, run Qdrant beside am and accept say you go operate second service.

Below roughly a hundred thousand vectors, measure the brute-force scan before you install anything. Exhaustive search with perfect recall and no build step no be compromise for that size. Na the correct answer, and if you reach for approximate index instead, you go need handle tuning and RAM pressure in exchange for milliseconds wey you no dey spend before.

FAQ

I need dedicated vector database, or Postgres dey enough?

If your data don already dey inside Postgres, pgvector go dey enough for much longer than most comparisons dey suggest. E dey store vectors for ordinary column, so filtered search na a WHERE clause and your existing backup dey cover the index. Move go dedicated store like Qdrant when vector workload need its own memory limit, or when you need payload filtering and scalar or product quantisation wey pgvector no provide.

How many vectors one VPS fit hold?

Calculate am instead of guessing, using number_of_vectors * dimensions * 4 bytes * 1.5. One million 768-dimension vectors na roughly 4.3 GiB, so 8 GB plan fit hold am with space for Postgres. One million 3072-dimension vectors na roughly 17 GiB and need much bigger plan. The thing wey affect this number pass na the dimension of your embedding model, so choose the model with the memory cost for mind.

Why my vector search slow when the index dey for the same machine?

Network no be your problem when everything dey for one box, so check the two things wey fit cause am. First, time the embedding call by itself, because generating the query vector for CPU often dey take much longer than the search itself. Second, check whether the index dey inside RAM. HNSW search dey jump randomly through graph, so once the graph spill go disk, every jump fit become disk read. Qdrant own guidance be say when you halve the vectors wey dey inside RAM, search latency roughly doubles.

I suppose build HNSW index at all?

No build am below roughly hundred thousand vectors. Exhaustive scan dey read n * d * 4 bytes for every query. That na 307 MB for 100,000 vectors of 768 dimensions. Modern CPU fit stream am within tens of milliseconds, with perfect recall and without any build step. Measure the scan for your own hardware first. Build the index when the measured scan time truly slow, no be because benchmark article talk so.

Wetin raising m really dey cost me?

Na build time and insert time, far more than memory. For 768 dimensions, moving from default m = 16 go m = 64 increase graph links from 128 to 512 bytes per vector. That na extra 384 bytes against 3072 bytes of vector data, so total memory go rise by about 12 percent. But every insert must find and link four times as many neighbours. Tune ef_search first, because e cost nothing to change and no rebuild dey needed.