Paritok token gateway: cheaper agent bills?
Paritok compresses the file reads and tool output your coding agent sends. The project claims 74% fewer tokens. Here is the mechanism and the break-even math.
What Paritok does to a request
Paritok is a token gateway: a proxy that sits between your coding agent and the model API and compresses each request before forwarding it. Your agent talks to http://127.0.0.1:8080 instead of the provider. The proxy shrinks the tool schemas, the file reads, the tool output and the older turns, sends the smaller payload upstream, and hands the reply back unchanged.
The provider bills you for what arrives at the provider, but the invoice depends on which tokens were removed, their cache pricing and how many turns the agent takes. That is the whole idea. It is a different claim from "your context lasts longer", and it is the reason this tool is interesting rather than just tidy.
The project is young. The gateway was open-sourced on 15 July 2026, and PyPI dates v1.3.14 to 6 October 2026 and v1.3.0 to 5 August. The weights and the gateway code are Apache 2.0. The compression model is a LoRA (low-rank adaptation) adapter on Qwen3-4B-Instruct-2507, distilled from a pool of 45,000 segments taken from real coding-agent trajectories, of which 40,606 passed teacher validation and became the training examples.
Why this is not context trimming
Trimming deletes for good. When an agent nears its context limit and drops the oldest turns, the file it read at turn 3 is gone. If it needs that file at turn 20, it reads the file again, so you pay for those tokens a second time. The saving was a loan.
Paritok replaces a segment with a shorter form plus a tag, [REF:id], and keeps the full text on the proxy. The model recovers a segment by calling read_original or expand_context. That changes the failure mode. A trimmer fails by forgetting, and it never tells you. A compressor fails by handing the model a lossy excerpt, and the model can ask for the original when the excerpt is not enough.
Filtered tool schemas remain recoverable, but the default adaptive filter drops unselected standard tools and drops unselected MCP tools unless its MCP relevance signal fires. When that signal fires, unselected MCP tools are stubbed. gateway_search_tools supports recovery, and the gateway also detects missing-tool messages and injects matching tools before retrying. This matters because a filter that permanently hides a tool changes what your agent is able to do, and you would learn about it through a task that quietly went wrong.
The three levers, and which one is free
Lever one is the tool-schema filter. Every request carries the whole tools array. On a Claude Code turn with no MCP (model context protocol) servers attached, the project measures that block at roughly 29,000 tokens; with MCP attached, it reports 44,000 to 65,000 tokens. The filter embeds the user's request and each tool description with BAAI/bge-small-en-v1.5, a 130 MB embedding model, keeps the tools that match, drops unselected standard tools, and stubs unselected MCP tools only when its MCP relevance signal fires. Otherwise, it drops those MCP tools too. In the project's 29,000-token example, the block drops to roughly 8,000 tokens. That embedding model runs on CPU.
Lever two is content compression, and this is the part that uses the 4B model, on a GPU or through Ollama on CPU. File reads, tool output and history are cut down to 25.7% of their original size. The model primarily selects spans rather than rewriting them, with limited rewriting allowed for such content as reasoning summaries and tool-call framing. The project's paper measures 96.0% of emitted identifiers, paths and numbers as appearing verbatim in the input in the teacher-generated training corpus; the trained model's held-out SWE-bench Lite output scores 96.2% on the same audit. That is where the 74% headline comes from. Read it carefully: 74% is the compression rate on the content that gets compressed, not the cut on your bill.
Lever three is history summarization. By default, once context exceeds 80% of the configured window, older turns are summarized while four recent turns stay intact so a long session keeps running instead of hitting the limit.
A GPU is optional for self-hosting. That is the most useful sentence on this page. pip install "paritok[toolselect]" gives you the tool filter on an ordinary CPU VPS, with no per-token fee. Try it before you rent a card.
What the project measured, and on whose harness
The data behind this chart
[
{
"label": "Paritok-4B-v1",
"compressed_to_pct": 25.7,
"quality_retained_pct": 86.5
},
{
"label": "gpt-4.1-mini",
"compressed_to_pct": 50.2,
"quality_retained_pct": 85.6
},
{
"label": "gpt-5",
"compressed_to_pct": 61.9,
"quality_retained_pct": 93.6
}
]These are the project's own published figures, measured on its own harness against SWE-bench Lite. Paritok-4B-v1 compresses content to 25.7% of original size while retaining 86.5% of the uncompressed solve rate. Using gpt-5 as the compressor retains more quality, 93.6%, but only compresses to 61.9%, and you would be paying frontier prices in order to save frontier prices.
Read the quality column honestly. Quality retained is the compressor solve rate divided by the uncompressed baseline solve rate. Retaining 86.5% means a 13.5% net reduction in solves relative to that baseline; it does not count the fraction of baseline-solved problems lost, because compressed runs can also solve different problems. The figure measures single-shot comprehension of compressed raw source with recall disabled and production edit recovery enabled; it does not measure the CPU tool-schema filter or the full multi-turn gateway.
The data behind this chart
[
{
"label": "Turn 1",
"saved_pct": 25
},
{
"label": "Turn 5",
"saved_pct": 39
},
{
"label": "Turn 12",
"saved_pct": 57
},
{
"label": "Turn 20",
"saved_pct": 63
}
]End-to-end saving grows as a session runs, because history accumulates and history is the thing being compressed. The project reports about 25% on a single turn, 39% by turn 5, and 63% by turn 20. Only turns 1 to 5 are measured; the README labels the later rows as projections. It also states where the growth stops: on a 200,000 token budget the per-turn content saving flattens at around 48,000 tokens, somewhere near turn 8 to 12, because once the context is full the history stops growing. The separate tool-filter saving adds approximately 21,000 tokens per turn, making the projected combined plateau approximately 69,000 tokens per turn. The widely quoted "past 85%" figure describes context-saturated sessions. That is the best case, and the same README says the saving plateaus toward a ceiling of about 72% under default settings, so do not plan around it.
Does a 24GB GPU pay for Paritok?
A GPU is optional, and the model fits on cards smaller than 24 GB. For an illustrative rental calculation, take $0.44 per hour for a 24 GB card. Left running all month, that is 730 hours, so $321. Run only during working hours, 8 hours a day for 22 days, and it is 176 hours, so $77.
Now turn the token cut into a dollar cut. The reduction applies to input tokens. Output tokens pass through the proxy untouched, so the proxy does not compress them. Price the change in fresh input, cache writes and cache reads separately, using your provider's rates, then compare the total API cost before and after. Even if input accounts for 80% of your dollar total, multiplying a token reduction by 0.8 does not establish your bill saving.
The data behind this chart
[
{
"label": "Always on (730 hours)",
"required_api_saving_usd": 321.2
},
{
"label": "Workday only (176 hours)",
"required_api_saving_usd": 77.44
}
]A card left running pays for itself once your measured monthly API saving exceeds $321.2, or $77.44 if you stop the instance outside working hours. An 85% token reduction alone does not establish a 68% bill saving, so it cannot establish a monthly agent-spend threshold. Divide the rental cost by your measured bill-saving fraction to get that threshold.
Two things make this better than the chart suggests. The model does not need 24 GB: the q4 build is about 2.5 GB and the f16 build about 8 GB, so a smaller card, or a GPU box you already run for something else, drops every number in that chart. And stopping the instance when nobody is coding is the single largest lever here, because it cuts the rent by about three quarters.
One thing makes it worse. The compression pass is real work. Every token the 4B model compresses is a token it must read, and every span it keeps is one it must write back out, which adds latency to each agent turn. On a card you rent by the hour that cost appears as waiting, not as a line on an invoice, so it is easy to miss until you feel it.
If you are weighing rented GPU hours against API tokens in general, the break-even between a GPU VPS and API tokens runs the same arithmetic for inference itself.
Running the Paritok gateway on a VPS
The package metadata on PyPI accepts Python 3.10 or newer, though the README badge says 3.11+. Ubuntu 24.04 ships Python 3.12, so a plain VPS image is enough for the CPU tool filter.
sudo apt update && sudo apt install -y python3-venv curl
python3 -m venv /opt/paritok/venv
source /opt/paritok/venv/bin/activate
pip install "paritok[proxy]==1.3.14"
pip install "paritok[toolselect]==1.3.14"Pin the version. PyPI dates v1.3.0 to 5 August 2026 and v1.3.14 to 6 October 2026, and a project moving at that pace renames things between releases: v1.3.0 itself renamed expand_context to read_original in its API. A bare pip install paritok, or a git clone of main, hands you a different gateway next week and leaves no record of which one produced the numbers you measured.
The default backend is Ollama. Pull the model, then give it the short name the proxy looks for.
ollama pull paritok/paritok-4b-v1
ollama cp paritok/paritok-4b-v1 paritok-4b-v1Because the local model writes out the spans it keeps for every segment it compresses, a compression pass that runs long shows up as a stalled agent turn, and Ollama's num_predict cap on output length is the knob that bounds it.
Write paritok.yaml next to it. use_gpu_server: false is what keeps the compression on your own hardware.
use_gpu_server: false
local_model:
base_url: http://localhost:11434/v1paritok proxy --port 8080 --config-file paritok.yamlparitok up is the shortcut for pulling the model if it is missing and starting the proxy on port 8080; Ollama must already be installed. Check the proxy before you point an agent at it.
curl http://127.0.0.1:8080/health
curl http://127.0.0.1:8080/stats/health returns a small JSON object containing "status":"ok" and a version string. /stats returns compression totals and the proxy's own estimate of what it saved. Treat that estimate as the proxy grading its own work, and confirm it against your provider's usage page.
For throughput rather than convenience, vLLM serves the adapter on top of the base model.
vllm serve Qwen/Qwen3-4B-Instruct-2507 \
--enable-lora \
--lora-modules paritok-4b-v1=paritok/paritok-4b-v1 \
--max-lora-rank 32 \
--port 8000The --max-lora-rank 32 flag is in the project's own command because the adapter is rank 32. Then set local_model.base_url to http://localhost:8000/v1 in paritok.yaml so the proxy talks to vLLM instead of Ollama.
Ollama is quicker to stand up. vLLM handles concurrent requests far better, which starts to matter as soon as more than one agent shares the box. The practical difference between Ollama and vLLM is what decides this one for you.
Point the agent at the proxy with the base URL environment variables.
export ANTHROPIC_BASE_URL=http://127.0.0.1:8080
export OPENAI_BASE_URL=http://127.0.0.1:8080Codex CLI ignores OPENAI_BASE_URL, so paritok up writes ~/.codex/config.toml for you when codex.enabled: true is set in paritok.yaml. Exporting the variable on its own leaves Codex talking straight to the provider, and the sign of that is a /stats counter that never moves while you work.
Keep the listener on 127.0.0.1, never on 0.0.0.0. The proxy forwards the credentials supplied with each request upstream, and its unauthenticated /stats endpoint exposes samples of original and compressed content, so an internet-facing listener exposes private context and local compression resources. Reach it from a laptop over an SSH tunnel or a VPN instead of opening the port.
Run it under systemd so it survives a reboot. Adjust the paths to match your install. Create the paritok user the unit runs as, and give it the install directory:
sudo useradd --system --home-dir /opt/paritok --shell /usr/sbin/nologin paritok
sudo chown -R paritok:paritok /opt/paritok[Unit]
Description=Paritok compression proxy
After=network-online.target
[Service]
User=paritok
WorkingDirectory=/opt/paritok
ExecStart=/opt/paritok/venv/bin/paritok proxy --port 8080 --config-file /opt/paritok/paritok.yaml
Restart=on-failure
[Install]
WantedBy=multi-user.targetEnable it with sudo systemctl enable --now paritok, then curl /health again. A unit that starts and immediately exits usually means the config file path is wrong, and journalctl -u paritok -n 50 prints the reason.
The hosted option, and what it costs you
The project also sells the compression as a service. Set use_gpu_server: true with an API key from paritok.com and the 4B model runs on its hardware. As of 2 November 2026 it bills $0.30 per million file tokens it compresses, from prepaid credit, and tool-schema filtering is free. New accounts get $5 of free credit with no card needed. The launch period during which the hosted service was free ended on 1 October 2026, so it is a paid service now. That removes the compression-model hosting work, while you still run the gateway.
It also means your prompts and the files your agent reads leave your machine and reach a third party before they reach your model provider. Self-hosting exists to avoid exactly that hop. Decide which of the two you are optimizing for before you set that flag, because the flag is a one-line change and the consequence is not.
How to measure your own before and after
The published numbers are the project's numbers, from the project's harness, on SWE-bench Lite. Your repository is not SWE-bench Lite. Measure your own.
- Run one normal week with no proxy in the path. Record fresh input, cache-write, cache-read and output tokens as separate lines from your provider's usage page, not as one dollar total.
- Run the next week with the proxy in front, doing the same kind of work.
- Compare the input categories at their actual prices and include any change in output cost. The proxy does not compress output, but changes in agent behaviour and turn count still affect how much output you buy.
- Count the tasks you had to redo. That is the quality half of the trade, and no dashboard anywhere reports it.
- Add the GPU hours to week two before you compare the totals.
Splitting input from output matters because the two are priced very differently and a compressor only touches one of them. As of 2 November 2026, Claude Sonnet 4.6 costs $3 per million input tokens and $15 per million output tokens, and a prompt-cache read is 10% of the input rate, $0.30 per million. The newer Claude Sonnet 5.5 costs $2 and $10, and its cache reads are 5% of the input rate, $0.10 per million. The gap between input and output token cost is what decides whether an input-side compressor is worth anything to you. Where Claude Code's tokens actually go tells you which part of your context is large enough to bother compressing.
Prompt caching complicates the tool-filter arithmetic in particular. The tool block sits at the front of the request, so after the first turn it is normally a cache hit; on Sonnet 4.6 that costs 10% of the input price. Cutting 21,000 tokens from a cached block saves 21,000 at $0.30 per million, about $0.006 per turn, rather than the $0.063 the uncached rate would suggest. The project keeps the filtered block frozen for the session so the cached prefix does not change. A filter that re-picked tools every turn would invalidate that prefix and could cost more than it saved.
What is still unverified
Every performance number above comes from the project itself. An independent synthetic-corpus compression check exists, but it explicitly does not test SWE-bench task success; the project's SWE-bench Lite solve-quality figures remain independently unconfirmed. With the gateway open-sourced in July 2026, there is very little operational history behind the code either. The compression rate and the quality-retained figure are both measured by the party that benefits from them looking good. That does not make them wrong. It makes them unconfirmed, and you should hold them differently from a number you produced yourself.
One documented behaviour is worth knowing before you blame your setup. The embedding model used by the tool filter loads on the first request rather than at startup, so the project documents a warm-up of 10 to 15 seconds, then around 15 ms per call afterwards. Send one throwaway request after the proxy starts and your first real agent turn will not appear to hang.
Four things you can settle yourself in an afternoon: whether the proxy starts and stays up, whether /stats moves while you work, whether your provider's input-token line actually falls, and whether the agent still finishes the work. Those decide it for your setup far better than any published benchmark.
On where this sits next to your other tooling: a self-hosted LiteLLM gateway routes and meters requests without altering their contents, so the two solve different problems and can be chained, with Paritok sitting closest to the agent. If the real goal is a smaller bill rather than this specific tool, the wider set of cost controls for an agent on a VPS includes several changes that cost nothing to try first.
FAQ
Does Paritok reduce my API bill or only my context usage?
It shrinks the request before it reaches the provider; the bill saving depends on the prices of the tokens removed and the agent's resulting turn count. The size of that reduction is smaller than the headline suggests. The 74% figure is the compression rate on the content being compressed. End to end, the project reports about 25% on a single turn and projects 63% by turn 20 as input-token savings. Output tokens pass through untouched, but the amount generated still depends on agent behaviour.
How much GPU do I need to self-host the compression model?
A GPU is optional: Ollama runs the model on CPU too. The q4 build is about 2.5 GB and the f16 build about 8 GB, so the model fits inside a 24 GB card with a great deal of room left over. A smaller card works, and it changes the break-even arithmetic in your favour. The tool-schema filter needs no GPU at all: it uses BAAI/bge-small-en-v1.5, a 130 MB embedding model that runs on CPU. Install paritok[toolselect] on an ordinary VPS and you get the tool-block reduction for the price of a little RAM.
What happens if the compressor removes something the agent needed?
Nothing is lost for good. The model deletes spans from what gets forwarded, but compressed segments carry a [REF:id] tag and the model recovers the full text with read_original or expand_context. The default tool filter drops unselected standard tools and drops unselected MCP tools unless its MCP relevance signal fires, in which case it stubs those MCP tools. Tool recovery uses gateway_search_tools and detection of missing-tool messages. The real risk is quieter than a missing file: the model works from a lossy excerpt and never realises it should ask for the original. The 86.5% quality-retained figure on SWE-bench Lite measures single-shot comprehension of compressed raw source with recall disabled and production edit recovery enabled; it does not measure the CPU tool-schema filter or the full multi-turn gateway.
Why does my first request take fifteen seconds?
The embedding model behind the tool filter loads on the first request instead of at startup. The project documents a warm-up of 10 to 15 seconds, then roughly 15 ms per call after that. Send one throwaway request with curl after starting the proxy, and the first real agent turn will not stall.
Should I use the hosted GPU server instead of self-hosting?
It removes compression-model hosting and any GPU rent, while you still maintain the gateway. As of 2 November 2026 it costs $0.30 per million file tokens compressed, paid from prepaid credit, with $5 of free credit for new accounts; the free launch period ended on 1 October 2026. It also sends your prompts and the files your agent reads to a third party before they reach your model provider. If you are self-hosting to keep code on infrastructure you control, that setting undoes the reason you started. Self-hosting keeps compression on your own box; compressed requests and provider credentials still go upstream to your model provider.