Vector database for VPS: pgvector, Qdrant or Chroma?

Wetin vector database for VPS really dey cost you

Running vector database for VPS (virtual private server) dey remove the problem wey every managed vendor dey sell solution for. Your application and index dey for the same machine, so search request dey pass through loopback socket instead of network. Wetin remain na the cost wey always be the real one: turning text into vectors. Two more costs dey behind am: the time to build the index and the RAM wey the index dey occupy as long as e dey serve requests.

This one change which decisions matter. Region and endpoint round trip no longer be your concern. The product of vector count, dimensions, and four bytes na your concern, because e determine whether the index go fit inside the memory wey you rent every month.

Where milliseconds dey really go for one box

Follow one similarity query through a self-hosted stack.

1. An embedding model dey turn the query text into vector. For CPU, this fit take tens to hundreds of milliseconds for short string. For GPU, e fit take single digits.
2. The vector dey go the store. Over loopback TCP or Unix domain socket, this dey take fraction of one millisecond.
3. The store dey scan its index and return the nearest rows.
4. Your code dey read the matching text and assemble prompt.

Step 1 usually get the biggest number for that list. Step 2 na the step wey hosted vendors dey compete on, and for one box e almost no dey exist. No guess the split. Time both ends for your own server.

curl http://127.0.0.1:11434/api/embed -s -o /dev/null \
  -w 'embed: %{time_total}s\n' \
  -d '{"model": "nomic-embed-text", "input": "how do I rotate the api key"}'

Then run \timing on inside psql before the search query. If the first command prints embed: 0.184312s and psql answers Time: 4.201 ms, index tuning na the wrong job: na the embedding model dey cause your latency. Run the embedding model locally with Ollama puts step 1 for the same CPU wey steps 2 and 3 dey use, so both halves dey compete for the same cores and same RAM. The ingest and retrieval loop wey dey around this store dey covered for the self-hosted RAG pipeline guide. RAG mean retrieval augmented generation: you search your own documents and paste the best matches inside prompt.

Under about a hundred thousand vectors, scan all of dem

Exhaustive scan dey compare query against every vector wey you store. Recall dey perfect by definition. E no need index or build step, and e no fit become out of date compared with your data.

The arithmetic go tell you when e stop make sense. Scan dey read n * d * 4 bytes for each query, where n na vector count and d na dimension. For 100,000 vectors with 768 dimensions, na 307 MB for each query. Modern CPU fit stream am within few tens of milliseconds. For 5 million vectors, na 15 GB for each query. At that point, e no really be query again.

So store the vectors for SQLite and do the comparison with NumPy.

import sqlite3, numpy as np

db = sqlite3.connect("docs.db")
db.execute("CREATE TABLE IF NOT EXISTS docs (id INTEGER PRIMARY KEY, body TEXT, vec BLOB)")

def add(body, vec):
    v = np.asarray(vec, dtype=np.float32)
    v /= np.linalg.norm(v)
    db.execute("INSERT INTO docs (body, vec) VALUES (?, ?)", (body, v.tobytes()))
    db.commit()

def search(query_vec, k=5):
    rows = db.execute("SELECT id, body, vec FROM docs").fetchall()
    mat = np.frombuffer(b"".join(r[2] for r in rows), dtype=np.float32).reshape(len(rows), -1)
    q = np.asarray(query_vec, dtype=np.float32)
    q /= np.linalg.norm(q)
    scores = mat @ q
    return [(rows[i][0], rows[i][1], float(scores[i])) for i in np.argsort(-scores)[:k]]

Both sides dey scale to unit length, so dot product na cosine similarity, and higher score mean say the match dey closer. Load mat once when the process start instead of once for each query. This remove SQLite read completely from the hot path.

Measure the time for your own machine before you reject this approach.

import time
t = time.perf_counter()
search(q)
print(f"{(time.perf_counter() - t) * 1000:.1f} ms")

The honest limits be these: na one process dey hold the complete matrix for RAM, and e no give you metadata filtering or proper support for concurrent writers. When any of these na why you no happy, move go another option. SQLite itself na serious server-side store, as this guide about SQLite for production explain, and if your real workload dey scan columns instead of serving rows, then this comparison between DuckDB and SQLite go be the more useful read.

pgvector, if you don already dey run Postgres

If your application already get Postgres database, pgvector add the least new surface. Na extension, e no be service. Vectors dey inside normal table beside the row wey dem describe, so filtered search na a WHERE clause instead of another system wey you need keep in sync.

Ubuntu 24.04 carry am for the universe component.

sudo apt update
sudo apt install -y postgresql-16-pgvector
sudo -u postgres psql -d yourdb -c 'CREATE EXTENSION vector;'

That package na pgvector 0.6.0 as of August 2026, and e dey well behind upstream. Iterative index scans especially need 0.8, so collect those ones from the PostgreSQL project's own repository.

sudo apt install -y postgresql-common
sudo /usr/share/postgresql-common/pgdg/apt.postgresql.org.sh
sudo apt install -y postgresql-17-pgvector

Replace 17 with your server major version, wey sudo -u postgres psql -tAc 'SHOW server_version' go print. If you install extension package wey dem build for wrong major version, CREATE EXTENSION go fail, because Postgres only dey look inside the share directory of the version wey dey run:

ERROR:  could not open extension control file "/usr/share/postgresql/16/extension/vector.control": No such file or directory

The schema na ordinary SQL with one new type.

CREATE TABLE chunks (
  id        bigserial PRIMARY KEY,
  doc_id    bigint NOT NULL,
  body      text   NOT NULL,
  embedding vector(768)
);

SELECT id, body FROM chunks
ORDER BY embedding <=> '[0.013, -0.021, 0.004]'
LIMIT 5;

<=> na cosine distance, <-> na L2 (Euclidean) distance, and <#> na negative inner product. Use the one wey dem train your embedding model for. If you pick wrong one, nothing go show error; na your results just go quietly worse.

If no index dey, that query na exact search across every row. Na the Postgres version of the brute force wey we talk about above, and e get the same perfect recall. If you raise max_parallel_workers_per_gather, more cores go work on am. Do this first, then create index later, because now you get recall baseline wey you fit use measure the index against.

Qdrant, when index don pass the database

Qdrant na dedicated vector store wey Rust write. E make sense when the index don big reach say you no want make e build dey compete with your application Postgres, or when you want payload filtering and quantisation wey pgvector no provide.

docker run -d --name qdrant \
  -p 127.0.0.1:6333:6333 -p 127.0.0.1:6334:6334 \
  -e QDRANT__SERVICE__API_KEY="$(openssl rand -hex 32)" \
  -v "$(pwd)/qdrant_storage:/qdrant/storage:z" \
  qdrant/qdrant

Port 6333 dey serve the REST (representational state transfer) API and dashboard for /dashboard, while 6334 dey serve gRPC. Two things important for public VPS. Qdrant documentation talk say the service dey run by default "with no encryption or authentication", and the -p 6333:6333 from quickstart dey bind to every interface. Docker fit publish am past ufw rule because e dey write im own forwarding rules. Bind am to 127.0.0.1 and set an API key. Qdrant instance wey public IP fit reach and wey no get key na public copy of your documents.

curl -s http://127.0.0.1:6333/collections -H "api-key: $QDRANT_API_KEY"

Healthy answer go look like {"result":{"collections":[]},"status":"ok","time":0.00002}. If {"status":{"error":"Unauthorized"}} come back, e mean say the header name or the key wrong. If nothing come back at all, e mean say the container no dey run or e bind somewhere else. Whether that container get the correct shape for your box na the normal stateful-service question, so the Docker versus host database comparison still apply here without change.

Chroma, and wetin e dey for

Chroma na the shortest way from nothing reach working retrieval demo.

pip install chromadb
chroma run --path /srv/chroma

That one dey listen for port 8000, and chromadb.HttpClient(host="localhost", port=8000) dey connect to am. Chroma get default embedding function, so first prototype no need separate model server at all.

Make we talk true about the trade-off. Chroma dey easy because e hide the decisions wey this guide dey discuss: which distance metric to use, and how much RAM the result go occupy. That one correct for prototype, but e wrong for the thing wey dem go page you about. If your data already dey Postgres, moving am to Chroma go add one process and one synchronisation problem just to solve problem wey pgvector no get.

Iwe ram index go need

Start from the raw vectors, because dem na the minimum size wey no tuning fit reduce.

bytes = number_of_vectors * dimensions * 4

Four bytes na one 32-bit float for each dimension. Qdrant capacity planning documentation add 1.5 multiplier for metadata and the temporary segments wey optimisation dey create:

memory_size = number_of_vectors * vector_dimension * 4 bytes * 1.5

Here na the formula result for one million vectors, using the dimensions wey real embedding models dey produce.

[
  {
    "label": "384 dims",
    "raw_gib": 1.43,
    "with_overhead_gib": 2.15
  },
  {
    "label": "768 dims",
    "raw_gib": 2.86,
    "with_overhead_gib": 4.29
  },
  {
    "label": "1024 dims",
    "raw_gib": 3.81,
    "with_overhead_gib": 5.72
  },
  {
    "label": "1536 dims",
    "raw_gib": 5.72,
    "with_overhead_gib": 8.58
  },
  {
    "label": "3072 dims",
    "raw_gib": 11.44,
    "with_overhead_gib": 17.17
  }
]

Na the formula output for gibibytes (GiB), no be measurement. Read dem as the memory space wey you need reserve. A 768-dimension model like nomic-embed-text for one million chunks need about 4.29 GiB. E fit inside 8 GB plan and still leave space for Postgres. The same corpus with 3072 dimensions need 17.17 GiB, so e no fit.

The important choice na the first column of that chart, no be the last one. If you halve the dimension, you halve every byte wey comes after am, permanently. A 768-dimension model wey score small lower for public leaderboard often na better engineering choice for VPS. pgvector's halfvec type then stores 16-bit floats, so e halve the bytes again, and e indexes through an expression:

CREATE INDEX ON chunks USING hnsw ((embedding::halfvec(768)) halfvec_cosine_ops);

Know one limit before you choose model. pgvector's vector type accepts up to 16,000 dimensions, but its HNSW and IVFFlat indexes only cover 2,000. If dimension pass that, you index a halfvec cast, wey reaches 4,000, or you no index am at all.

Wetin m and ef_construction cost for build time

HNSW (hierarchical navigable small world) na the index wey pgvector and Qdrant both dey use. Na layered graph. Every vector na node wey get links to nearby nodes, and search dey hop through those links go query, instead of reading everything.

m na how many links each node dey keep. Faiss documentation talk say HNSW memory na (d * 4 + m * 2 * 4) bytes per vector, and e recommend make you keep m between 4 and 64. Run that for 768 dimensions over one million vectors.

[
  {
    "label": "m = 8",
    "link_bytes_per_vector": 64,
    "total_gib": 2.92
  },
  {
    "label": "m = 16 (default)",
    "link_bytes_per_vector": 128,
    "total_gib": 2.98
  },
  {
    "label": "m = 32",
    "link_bytes_per_vector": 256,
    "total_gib": 3.1
  },
  {
    "label": "m = 64",
    "link_bytes_per_vector": 512,
    "total_gib": 3.34
  }
]

The useful surprise na how big that difference be. If you move from default m = 16 to m = 64, e add 512 bytes of links per vector against 3072 bytes of vector data. So total go move from 2.98 GiB to 3.34 GiB. That na about 12 percent. For these dimensions, m no be where your memory dey go. Na the vectors.

The real cost of m na build time and insert time, because to place one node, system must find and link that number of neighbours. ef_construction na the size of the candidate list wey builder dey check as e dey place each node. If you increase am, graph go better but build go slower. E no change the finished index size at all.

SET maintenance_work_mem = '4GB';
SET max_parallel_maintenance_workers = 7;
CREATE INDEX ON chunks USING hnsw (embedding vector_cosine_ops)
  WITH (m = 16, ef_construction = 64);

maintenance_work_mem na the setting wey decide whether build go take minutes or hours, because pgvector dey assemble the graph for memory when e fit. When e no fit again, pgvector go show this:

NOTICE:  hnsw graph no longer fits into maintenance_work_mem after 100000 tuples
DETAIL:  Building will take significantly more time.
HINT:  Increase maintenance_work_mem to speed up builds.

That notice na the most useful line wey pgvector dey print. E mean say build don enter much slower path. So cancel am, raise the setting pass the RAM figure wey you calculate above, then start again. Monitor progress from another session:

SELECT phase, round(100.0 * blocks_done / nullif(blocks_total, 0), 1) AS pct
FROM pg_stat_progress_create_index;

HNSW go report initializing and later loading tuples. If build stay for low percentage for long time while disk no busy, na the maintenance_work_mem problem, no be query wey hang.

Two facts wey you suppose plan for. pgvector README talk say HNSW "has slower build times and uses more memory" than IVFFlat, but e gives better speed-versus-recall trade-off. HNSW fit dey created for empty table, but IVFFlat must run k-means over representative data first. So if you build IVFFlat for empty table, recall go poor. For fresh schema, HNSW na the one wey you fit create ahead of time.

ef_search: na knob wey you tune after the build

m and ef_construction don freeze inside the index. ef_search no freeze. E set how many candidates the search go keep while e dey walk through the graph, and you fit change am per session or per query.

SET hnsw.ef_search = 100;
SELECT id, body FROM chunks ORDER BY embedding <=> $1 LIMIT 5;

The default na 40. Raise am, recall go increase, and latency go increase too. Lower am, both go reduce. Na this only recall control you fit change without rebuild, so tune am against fixed set of queries wey you already know their correct answers, then stop when recall stop improving.

One trap dey. ef_search no work well with selective WHERE clause, because the index dey return fixed number of candidates and dem apply the filter afterwards. Filter wey reject most rows fit leave you with fewer than LIMIT results, even when matching rows still dey inside the table. pgvector 0.8 solve this with iterative scans:

SET hnsw.iterative_scan = relaxed_order;

The index go scan again for more candidates until e satisfy the limit, up to hnsw.max_scan_tuples, wey default na 20000. strict_order maintain exact distance ordering, but e cost more. Na this feature the Ubuntu 0.6.0 package no get, and missing rows under a filter na how you go notice am.

Why index gree fit inside RAM

HNSW search na walk over graph. Every hop dey read node wey dey somewhere different from the previous one, so the access pattern almost random, and read-ahead no help. As long as graph dey inside RAM, every hop na memory reference. Once e no dey there again, one hop fit become disk read, and search wey touch few hundred nodes fit cause few hundred reads.

Qdrant documentation explain am like this: "if you store half as many vectors in RAM, search latency will roughly double." Plan with this sentence for mind.

When index really no go fit, every option na trade-off wey you suppose choose deliberately.

• Memory-map the vectors so operating system fit cache hot pages and leave cold ones for disk. You need fast NVMe (non-volatile memory express) storage underneath before this go tolerable.
• Quantise, by storing each dimension as one byte instead of four. This reduce vector bytes by four, with small but measurable recall cost.
• Cast to halfvec for pgvector, wey cut the bytes by half and cause less recall loss than one-byte quantisation.
• Embed with smaller model. This na the cheapest fix and the one people dey skip, because e mean say you must re-embed the corpus.

Wahala wey fit happen and strings wey you go see

could not open extension control file. pgvector package for the Postgres major version wey dey run no dey installed. Print the version with sudo -u postgres psql -tAc 'SHOW server_version' and install the matching postgresql-NN-pgvector.

ERROR: expected 768 dimensions, not 1536. Column type and model no agree. You change embedding models but you no re-embed. No partial fix dey here, because vectors from two different models no comparable at all, so you must regenerate every row.

Query dey slow and EXPLAIN show sequential scan. Index operator class and query operator no match. vector_cosine_ops only support <=>. Run EXPLAIN ANALYZE for the query and look for Index Scan using ... on chunks. If you see Seq Scan on chunks instead, rebuild the index with the operator class wey match the operator wey you actually dey query with.

Rows dey fewer than LIMIT, and WHERE clause dey present. Na the filtering trap wey we talk about above. Increase hnsw.ef_search, or move to pgvector 0.8 and set hnsw.iterative_scan.

Index build end with the process gone and no error for psql. If you set maintenance_work_mem to most of the machine, while shared_buffers and your application still need memory, kernel out-of-memory killer fit terminate the process. sudo dmesg -T | grep -i 'killed process' go show the line wey name postgres. Reduce the setting, or build the index on a bigger plan and restore the dump.

Wey one to pick

If you already dey run Postgres and you get less than a few million vectors, use pgvector. The index dey beside the data, filtering na a WHERE clause, and your existing backup already cover am. If the index big reach where e need im own memory ceiling, or you need heavy payload filtering, run Qdrant beside am and accept say you go operate second service.

Below roughly a hundred thousand vectors, measure the brute-force scan before you install anything. Exhaustive search with perfect recall and no build step no be compromise for that size. Na the correct answer, and if you reach for approximate index instead, you go need handle tuning and RAM pressure in exchange for milliseconds wey you no dey spend before.

FAQ

Dedicated vector database I need, or Postgres enough?

If your data don already dey inside Postgres, pgvector go dey enough for much longer than most comparisons dey suggest. E dey store vectors for ordinary column, so filtered search na WHERE clause and your existing backup dey cover the index. Move go dedicated store like Qdrant when vector workload need its own memory ceiling, or when you need payload filtering and quantisation wey pgvector no provide.

How many vectors one VPS fit hold?

Calculate am instead of guessing, using number_of_vectors * dimensions * 4 bytes * 1.5. One million 768-dimension vectors na roughly 4.3 GiB, so 8 GB plan fit hold am with space for Postgres. One million 3072-dimension vectors na roughly 17 GiB and need much bigger plan. The thing wey change this number pass na the dimension of your embedding model, so choose the model with the memory cost for mind.

Why vector search slow when the index dey for the same machine?

Network no be your problem when everything dey for one box, so check the two things wey fit cause am. First, measure the embedding call by itself, because generating the query vector for CPU often dey take much longer than the search itself. Second, confirm say the index dey inside RAM. HNSW search dey jump randomly through graph, so when the graph spill go disk, every jump fit become disk read. Qdrant own guidance say when you halve the vectors wey dey inside RAM, search latency fit roughly double.

I suppose build HNSW index at all?

No build am when you get below roughly hundred thousand vectors. Exhaustive scan dey read n * d * 4 bytes per query. That na 307 MB for 100,000 vectors of 768 dimensions. Modern CPU fit stream am within tens of milliseconds, with perfect recall and without build step. Measure the scan for your own hardware first. Build the index when the measured scan time truly slow, no be because benchmark article talk say make you do am.

Wetin raising m actually dey cost me?

E dey cost build time and insert time, much more than memory. For 768 dimensions, moving from default m = 16 go m = 64 adds 512 bytes of graph links per vector against 3072 bytes of vector data. So total memory go rise by about 12 percent. But every insert must find and link four times as many neighbours. Tune ef_search first, because e cost nothing to change and e no need rebuild.