Agent memory types and what they cost

What the three agent memory types are

Agent memory types split three ways, and each one lands differently on hardware you pay for: semantic memory holds facts, episodic memory holds what happened, and procedural memory holds how to do a job. The table below defines each one with a server example. Everything after it is the half that usually goes unwritten, which is what each type costs to store and what it costs to rebuild.

[
  {
    "label": "Semantic",
    "what_it_holds": "Facts the agent should treat as currently true",
    "server_example": "The database listens on 10.8.0.4:5432 and the nightly dump runs at 03:15 UTC"
  },
  {
    "label": "Episodic",
    "what_it_holds": "A record of one past event or session",
    "server_example": "On 2026-08-11 the deploy failed because the disk was full, and rotating logs fixed it"
  },
  {
    "label": "Procedural",
    "what_it_holds": "How to carry out a task, as steps the agent can run",
    "server_example": "The restore runbook: stop the service, load the dump, run migrations, start the service"
  }
]

The names are borrowed from human psychology and the borrowing is loose. The split earns its place for a practical reason: the three kinds have different sizes and different repair paths, so putting all of them in one vector store makes each of them worse.

Semantic memory is small, and you will want to edit it by hand

A few hundred facts about your own servers is a few tens of kilobytes of text. Storage is not the problem here. Correction is. A wrong fact in semantic memory is wrong in every answer the agent gives after it, so the store has to let you find one fact by name, change it, and be certain the old value is gone.

That points at a keyed store: a Postgres table with a primary key, or a directory of small markdown files in git. Both let you run one query, see the value, and edit it in place. Similarity search does not give you that, because you retrieve by resemblance instead of by key. "Change the database port" becomes "find every chunk that mentions the database port", and you cannot prove you found them all. Keep the facts keyed. Embed them as well if you also want loose recall, but treat the keyed copy as the truth.

Stale facts do not announce themselves. The port moves and the row stays, so the agent keeps answering with a number that was correct in June. A staleness and pruning policy for agent memory is the other half of this page, and it is much cheaper to design while the table is still small.

Why episodic memory grows without bound

Episodic memory is a log, and logs grow. Every session, every tool call and every failed command is a candidate episode. An agent that writes one row per turn will write far more rows in a month than anyone will ever read, and disk is not the only cost: every embedded episode also joins the index that search has to walk.

Decide the retention rule on the day you create the table, while deleting is still free. Two questions settle most of it. First, what is worth writing at all: a summary of a session usually is, the full output of one ls -la usually is not. Second, how long each class of episode lives: something like raw episodes for 30 days and session summaries for a year.

Give every episode row a created_at timestamp and a source column. Without created_at you cannot delete by age. Without source you cannot delete everything that arrived from one bad origin, which is exactly what you need on the day a web page or a ticket turns out to have been writing instructions into memory.

DELETE FROM episodes WHERE created_at < now() - interval '30 days';

Run that from a systemd timer, then watch the row count and the table size actually move afterwards. A retention policy that nobody executes is a comment.

psql -d agentmem -c "SELECT count(*) FROM episodes;"
psql -d agentmem -c "SELECT pg_size_pretty(pg_total_relation_size('episodes'));"

Procedural memory belongs in a repository

Procedural memory is how the agent does a job: a shell script, a skill file, a runbook with numbered steps. That is code, and code belongs where code lives, in a git repository with review, versions and a diff you can read.

Store a runbook as embedded chunks and you get an approximate copy back. Retrieval returns the chunks that scored highest, so the agent can act on step 2 and step 5 while step 3 never surfaced, and nothing records which version of the procedure ran. In git, git log answers both questions. The storage cost is near zero, which is the other reason not to pay vector prices for it.

What an embedding actually costs on disk

[
  {
    "label": "bge-small-en-v1.5 (384 dims)",
    "bytes_per_vector": 1544,
    "mib_per_100k_rows": 147.2
  },
  {
    "label": "bge-base-en-v1.5 (768 dims)",
    "bytes_per_vector": 3080,
    "mib_per_100k_rows": 293.7
  },
  {
    "label": "bge-large-en-v1.5 (1024 dims)",
    "bytes_per_vector": 4104,
    "mib_per_100k_rows": 391.4
  },
  {
    "label": "text-embedding-3-small (1536 dims)",
    "bytes_per_vector": 6152,
    "mib_per_100k_rows": 586.7
  },
  {
    "label": "text-embedding-3-large (3072 dims)",
    "bytes_per_vector": 12296,
    "mib_per_100k_rows": 1172.6
  }
]

pgvector stores a vector as 4 bytes per dimension plus an 8 byte header, so this arithmetic is fixed and you can plan for it before loading anything. One 384 dimension vector is 1544 bytes, which makes 100,000 chunks 147.2 MiB of vectors. The same corpus embedded at 3,072 dimensions is 1172.6 MiB, at 12296 bytes per row. Same text, close to eight times the storage.

That is the vector column alone. The chunk text, the primary key, row overhead and the index all sit on top, and the index is the part people forget. HNSW (hierarchical navigable small world, the graph index pgvector builds) keeps its own copy of the vectors it links, so an indexed store is comfortably more than double the figures above. Measure yours instead of guessing.

SELECT pg_size_pretty(pg_total_relation_size('memories')) AS total,
       pg_size_pretty(pg_relation_size('memories_embedding_idx')) AS idx;

RAM decides whether search feels fast, because the graph is quick to walk only while it is in memory. When the index is larger than the memory Postgres can hold it in, searches start reading from disk and latency rises. The build has its own limit, maintenance_work_mem, and when the graph outgrows it the build says so and slows down:

NOTICE:  hnsw graph no longer fits into maintenance_work_mem after 61440 tuples
HINT:  Increase maintenance_work_mem to speed up builds.

Two levers shrink the same corpus. Pick a smaller model, since 384 dimensions costs a quarter of 1,536 dimensions and for finding your own notes again the accuracy difference is often small enough to accept. Or store half precision: the halfvec type takes 2 bytes per dimension plus the same 8 byte header, which nearly halves the column and its index together.

One limit is worth knowing before you choose a model. As of August 2026 a vector column can be indexed up to 2,000 dimensions, so a 3,072 dimension embedding is accepted by the column and refused by the index:

ERROR:  column cannot have more than 2000 dimensions for hnsw index

halfvec indexes up to 4,000 dimensions, so the usual fix is to index the cast:

CREATE INDEX ON memories USING hnsw ((embedding::halfvec(3072)) halfvec_cosine_ops);

When pgvector beats a separate memory service

If the box already runs Postgres, vectors are one package and one statement.

psql -V
sudo apt install postgresql-16-pgvector

CREATE EXTENSION vector;

The number in the package name is your Postgres major version, 16 on Ubuntu 24.04, so read psql -V before you type it.

Keeping memory in that database buys you one backup covering memory and application data at the same moment, one connection pool, and transactions: a fact and the row it describes commit together or fail together. A separate service cannot promise that.

Move to a dedicated memory service when one of these is true. Search load competes with your application and wants its own machine. Several agents on several hosts share one memory. Or you want the extraction and deduplication logic that a finished product ships with, which is the argument for a self-hosted Mem0 memory server. For one agent and a corpus in the low millions of chunks, pgvector on the box you already run is less to operate and less to break. The engine choice itself, and what each engine wants in RAM, is covered in running a vector database on a VPS.

What re-embedding costs when you change the model

Vectors from two different models are not comparable, so you cannot embed new memories with a new model and leave the old rows alone. A mixed table returns nonsense, because a distance computed between two different coordinate systems is a number with no meaning. Changing model means re-embedding the whole corpus.

That bill has four parts: the tokens (an API charge, or CPU and GPU time on your own box), the wall clock time while it runs, the disk for both columns at once during the backfill, and the index rebuild at the end. The safe order is add a new column, backfill it in batches, swap the query, then drop the old column and its index.

Measure the rate on your own hardware rather than trusting a published figure, because CPU-only embedding on a small VPS runs far slower than the same model on a GPU. Time one representative chunk, then multiply by the corpus size.

ollama pull nomic-embed-text
time curl -s http://localhost:11434/api/embed \
  -d '{"model": "nomic-embed-text", "input": "one representative chunk of your corpus"}' > /dev/null

A local embedding model also puts its weights on the same disk as the memory store, and where Ollama stores pulled models explains where that space goes.

One requirement makes all of this possible: keep the source text next to every vector. A store holding only vectors cannot be re-embedded at all, because there is nothing left to feed the new model. If you cannot answer "what text produced this row", your migration path is a full rebuild from wherever the text first came from.

What to watch once memory is running

Cost is not the only thing that changes as the store fills. Old facts drift out of date and old episodes crowd out useful results, which is the pruning problem again. A memory store is also a writable input to the agent's future behaviour, so anything allowed to write into it can steer the agent later. If text from web pages or tickets reaches memory, read how agent memory poisoning works before you widen what may write. Retrieval size drives token spend on every single request too, and that is where keeping an agent's running cost predictable begins.

FAQ

Do I need a vector database for agent memory?

Not for facts. Semantic memory is small and you need to correct it by name, so a keyed table or a directory of markdown files in git serves it better, because you can see one value and edit it. Embeddings earn their cost when you must retrieve by meaning across a corpus too large to list, which is usually episodic memory and documents. If you already run Postgres, CREATE EXTENSION vector covers that without adding another service to operate.

How much disk will an agent memory store use?

The vectors are predictable, at 4 bytes per dimension plus an 8 byte header in pgvector. At 768 dimensions that is 293.7 MiB per 100,000 rows, and at 384 dimensions it is 147.2 MiB. Then add the chunk text, the row overhead, and an HNSW index that keeps its own copy of the vectors, so budget at least double the vector figure and measure the real number with pg_total_relation_size.

Where should procedural memory live?

In a git repository, as scripts or skill files the agent runs directly. A procedure needs exact recall and a version history, and similarity search gives you neither. A chunked runbook comes back with the highest scoring pieces, which can mean step 2 and step 5 arrive while step 3 is missing, and nothing records which version ran.

What does changing the embedding model cost?

A full re-embedding of the corpus, because vectors from different models cannot be compared against each other. Budget the token charge or GPU time, the disk for the old and new columns at the same time, and the index rebuild. Add the new column, backfill in batches, swap the query, then drop the old column. All of this depends on having kept the source text beside each vector.