By Paritok token gateway: cheaper agent bills?
Paritok compresses the file reads and tool output your coding agent sends. The project claims 74% fewer tokens. Here is the mechanism and the break-even math.
What Paritok does to a request
Paritok is a token gateway: a proxy that sits between your coding agent and the model API and compresses each request before forwarding it. Your agent talks to http://127.0.0.1:8080 instead of the provider. The proxy rewrites the tool schemas, the file reads, the tool output and the older turns, sends the smaller payload upstream, and hands the reply back unchanged.
The provider bills you for what arrives at the provider, so a smaller payload is a smaller invoice. That is the whole idea. It is a different claim from "your context lasts longer", and it is the reason this tool is interesting rather than just tidy.
The project is young. Its first public tags are dated July 2026 and the current tag is v1.3.0, dated 5 August 2026. The weights and the gateway code are Apache 2.0. The compression model is a LoRA (low-rank adaptation) adapter on Qwen3-4B-Instruct-2507, trained on 45,000 teacher-distilled samples taken from real coding-agent trajectories.
Why this is not context trimming
Trimming deletes. When an agent nears its context limit and drops the oldest turns, the file it read at turn 3 is gone. If it needs that file at turn 20, it reads the file again, so you pay for those tokens a second time. The saving was a loan.
Paritok replaces a segment with a shorter form plus a tag, [REF:id], and keeps the full text on the proxy. The model recovers a segment by calling read_original or expand_context. That changes the failure mode. A trimmer fails by forgetting, and it never tells you. A compressor fails by handing the model a lossy summary, and the model can ask for the original when the summary is not enough.
The tool filter behaves the same way. Filtered tool schemas are stubbed rather than removed, and the model recovers one by calling gateway_search_tools. This matters because a filter that permanently hides a tool changes what your agent is able to do, and you would learn about it through a task that quietly went wrong.
The three levers, and which one is free
Lever one is the tool-schema filter. Every request carries the whole tools array. On a Claude Code turn with a few MCP (model context protocol) servers attached, the project measures that block at roughly 29,000 tokens. The filter embeds the user's request and each tool description with BAAI/bge-small-en-v1.5, a 130 MB embedding model, keeps the tools that match, and stubs the rest. The block drops to roughly 8,000 tokens. That embedding model runs on CPU.
Lever two is content compression, and this is the part that needs the 4B model on a GPU. File reads, tool output and history are rewritten down to 25.7% of their original size. That is where the 74% headline comes from. Read it carefully: 74% is the compression rate on the content that gets compressed, not the cut on your bill.
Lever three is history summarization. Once the context budget fills, turns beyond the recent window are summarized so a long session keeps running instead of hitting the limit.
Only lever two needs a GPU. That is the most useful sentence on this page. pip install "paritok[toolselect]" gives you the tool filter on an ordinary CPU VPS, and it is the half of the product that costs you nothing per month. Try it before you rent a card.
What the project measured, and on whose harness
The data behind this chart
[
{
"label": "Paritok-4B-v1",
"compressed_to_pct": 25.7,
"quality_retained_pct": 86.5
},
{
"label": "gpt-4.1-mini",
"compressed_to_pct": 50.2,
"quality_retained_pct": 85.6
},
{
"label": "gpt-5",
"compressed_to_pct": 61.9,
"quality_retained_pct": 93.6
}
]These are the project's own published figures, measured on its own harness against SWE-bench Lite. Paritok-4B-v1 compresses content to 25.7% of original size while retaining 86.5% of the uncompressed solve rate. Using gpt-5 as the compressor retains more quality, 93.6%, but only compresses to 61.9%, and you would be paying frontier prices in order to save frontier prices.
Read the quality column honestly. Retaining 86.5% of the solve rate means the compressed runs failed problems that the uncompressed runs solved, close to one solve in seven. On a benchmark that is a number in a table. On your repository it is a task you run twice.
The data behind this chart
[
{
"label": "Turn 1",
"saved_pct": 25
},
{
"label": "Turn 5",
"saved_pct": 39
},
{
"label": "Turn 12",
"saved_pct": 57
},
{
"label": "Turn 20",
"saved_pct": 63
}
]End-to-end saving grows as a session runs, because history accumulates and history is the thing being compressed. The project reports about 25% on a single turn, 39% by turn 5, and 63% by turn 20. It also states where the growth stops: on a 200,000 token budget the absolute saving flattens at around 48,000 tokens per turn, somewhere near turn 8 to 12, because once the context is full the history stops growing. The widely quoted "past 85%" figure describes context-saturated sessions. That is the best case, so do not plan around it.
Does a 24GB GPU pay for Paritok?
A 24 GB card is the usual rental unit for a model this size. As of 7 August 2026, the median published on-demand rate for an RTX 4090 with 24 GB was $0.44 per hour, with the cheapest listings near $0.20. Take $0.44. Left running all month, that is 730 hours, so $321. Run only during working hours, 8 hours a day for 22 days, and it is 176 hours, so $77.
Now turn the token cut into a dollar cut. The reduction applies to input tokens. Output tokens pass through the proxy untouched, so they do not move at all. Assume input tokens are 80% of your dollar total, which is normal for a coding agent, and check that assumption against your own bill. Your dollar saving is then the token reduction multiplied by 0.8.
The data behind this chart
[
{
"label": "Turn 5 (39% saved)",
"bill_always_on_usd": "1,030",
"bill_workday_only_usd": 248
},
{
"label": "Turn 20 (63% saved)",
"bill_always_on_usd": 637,
"bill_workday_only_usd": 154
},
{
"label": "Saturated (85% saved)",
"bill_always_on_usd": 472,
"bill_workday_only_usd": 114
}
]At the saturated-session figure of 85%, you keep 68% of the bill, so a card left running pays for itself once your monthly agent spend passes about $472, or about $114 if you stop the instance outside working hours. At the turn-20 figure of 63%, those become $637 and $154. At the turn-5 figure of 39%, which is what short sessions actually look like, you need about $1,030 a month before the card is worth renting at all.
Two things make this better than the table suggests. The model does not need 24 GB: the q4 build is about 2.5 GB and the bf16 build about 8 GB, so a smaller card, or a GPU box you already run for something else, drops every number in that chart. And stopping the instance when nobody is coding is the single largest lever here, because it cuts the rent by about three quarters.
One thing makes it worse. The compression pass is real work. Every token the 4B model compresses is a token it must read and then write, which adds latency to each agent turn. On a card you rent by the hour that cost appears as waiting, not as a line on an invoice, so it is easy to miss until you feel it.
If you are weighing rented GPU hours against API tokens in general, the break-even between a GPU VPS and API tokens runs the same arithmetic for inference itself.
Running the Paritok gateway on a VPS
Python 3.10 or newer is required. Ubuntu 24.04 ships Python 3.12, so a plain VPS image is enough for the CPU-only half.
sudo apt update && sudo apt install -y python3-venv curl
python3 -m venv /opt/paritok/venv
source /opt/paritok/venv/bin/activate
pip install "paritok[proxy]==1.3.0"
pip install "paritok[toolselect]==1.3.0"Pin the version. The repository tagged v1.2.8 on 29 July 2026 and v1.3.0 on 5 August 2026, and a project moving at that pace renames config keys between releases. A bare pip install paritok, or a git clone of main, hands you a different gateway next week and leaves no record of which one produced the numbers you measured.
The default backend is Ollama. Pull the model, then give it the short name the proxy looks for.
ollama pull paritok/paritok-4b-v1
ollama cp paritok/paritok-4b-v1 paritok-4b-v1Write paritok.yaml next to it. use_gpu_server: false is what keeps the compression on your own hardware.
use_gpu_server: false
local_model:
base_url: http://localhost:11434paritok proxy --port 8080 --config-file paritok.yamlparitok up is the shortcut for all of the above: it pulls the model if it is missing and starts the proxy on port 8080. Check the proxy before you point an agent at it.
curl http://127.0.0.1:8080/health
curl http://127.0.0.1:8080/stats/health returns a small JSON object containing "status":"ok" and a version string. /stats returns compression totals and the proxy's own estimate of what it saved. Treat that estimate as the proxy grading its own work, and confirm it against your provider's usage page.
For throughput rather than convenience, vLLM serves the adapter on top of the base model.
vllm serve Qwen/Qwen3-4B-Instruct-2507 \
--enable-lora \
--lora-modules paritok-4b-v1=paritok/paritok-4b-v1 \
--port 8000Ollama is quicker to stand up. vLLM handles concurrent requests far better, which starts to matter as soon as more than one agent shares the box. The practical difference between Ollama and vLLM is what decides this one for you.
Point the agent at the proxy with the base URL environment variables.
export ANTHROPIC_BASE_URL=http://127.0.0.1:8080
export OPENAI_BASE_URL=http://127.0.0.1:8080Codex CLI ignores OPENAI_BASE_URL, so the project writes ~/.codex/config.toml for you when codex.enabled: true is set in paritok.yaml. Exporting the variable on its own leaves Codex talking straight to the provider, and the sign of that is a /stats counter that never moves while you work.
Keep the listener on 127.0.0.1, never on 0.0.0.0. The proxy forwards your provider API key upstream, so a proxy reachable from the internet is an open relay for that key: whoever finds the port spends your money without ever seeing the key itself. Reach it from a laptop over an SSH tunnel or a VPN instead of opening the port.
Run it under systemd so it survives a reboot. Adjust the paths to match your install.
[Unit]
Description=Paritok compression proxy
After=network-online.target
[Service]
User=paritok
WorkingDirectory=/opt/paritok
ExecStart=/opt/paritok/venv/bin/paritok proxy --port 8080 --config-file /opt/paritok/paritok.yaml
Restart=on-failure
[Install]
WantedBy=multi-user.targetEnable it with sudo systemctl enable --now paritok, then curl /health again. A unit that starts and immediately exits usually means the config file path is wrong, and journalctl -u paritok -n 50 prints the reason.
The hosted option, and what it costs you
The project also sells the compression as a service. Set use_gpu_server: true with an API key and the 4B model runs on its hardware, priced at $0.30 per million tokens processed, free through the end of August 2026 according to its own documentation. That removes the GPU rent and all of the operations work above.
It also means your prompts and the files your agent reads leave your machine and reach a third party before they reach your model provider. Self-hosting exists to avoid exactly that hop. Decide which of the two you are optimizing for before you set that flag, because the flag is a one-line change and the consequence is not.
How to measure your own before and after
The published numbers are the project's numbers, from the project's harness, on SWE-bench Lite. Your repository is not SWE-bench Lite. Measure your own.
- Run one normal week with no proxy in the path. Record input tokens, cache-read tokens and output tokens as separate lines from your provider's usage page, not as one dollar total.
- Run the next week with the proxy in front, doing the same kind of work.
- Compare the input and cache-read lines. Output should be roughly flat, because nothing compresses it. If output moved a lot, something other than the proxy changed.
- Count the tasks you had to redo. That is the quality half of the trade, and no dashboard anywhere reports it.
- Add the GPU hours to week two before you compare the totals.
Splitting input from output matters because the two are priced very differently and a compressor only touches one of them. As of August 2026, Claude Sonnet 4.6 costs $3 per million input tokens and $15 per million output tokens, and a prompt-cache read is 10% of the input rate, $0.30 per million. The gap between input and output token cost is what decides whether an input-side compressor is worth anything to you. Where Claude Code's tokens actually go tells you which part of your context is large enough to bother compressing.
Prompt caching complicates the tool-filter arithmetic in particular. The tool block sits at the front of the request, so after the first turn it is normally a cache hit at 10% of the input price. Cutting 21,000 tokens from a cached block saves 21,000 at $0.30 per million, about $0.006 per turn, rather than the $0.063 the uncached rate would suggest. The project keeps the filtered block frozen for the session so the cached prefix does not change. A filter that re-picked tools every turn would invalidate that prefix and cost more than it saved.
What is still unverified
Every performance number above comes from the project itself. There is no independent reproduction of the SWE-bench Lite results, and with the first tags dated July 2026 there is very little operational history behind the code either. The compression rate and the quality-retained figure are both measured by the party that benefits from them looking good. That does not make them wrong. It makes them unconfirmed, and you should hold them differently from a number you produced yourself.
One documented behaviour is worth knowing before you blame your setup. The embedding model used by the tool filter loads on the first request rather than at startup, so the project documents a warm-up of 10 to 15 seconds, then around 15 ms per call afterwards. Send one throwaway request after the proxy starts and your first real agent turn will not appear to hang.
Four things you can settle yourself in an afternoon: whether the proxy starts and stays up, whether /stats moves while you work, whether your provider's input-token line actually falls, and whether the agent still finishes the work. Those decide it for your setup far better than any published benchmark.
On where this sits next to your other tooling: a self-hosted LiteLLM gateway routes and meters requests without altering their contents, so the two solve different problems and can be chained, with Paritok sitting closest to the agent. If the real goal is a smaller bill rather than this specific tool, the wider set of cost controls for an agent on a VPS includes several changes that cost nothing to try first.
FAQ
Does Paritok reduce my API bill or only my context usage?
It reduces the bill, because the proxy rewrites the request before it reaches the provider and the provider charges for what it receives. The size of that reduction is smaller than the headline suggests. The 74% figure is the compression rate on the content being compressed. End to end, the project reports about 25% on a single turn and 63% by turn 20, and only input tokens move at all. Output tokens pass through untouched.
How much GPU do I need to self-host the compression model?
The q4 build is about 2.5 GB and the bf16 build about 8 GB, so the model fits inside a 24 GB card with a great deal of room left over. A smaller card works, and it changes the break-even arithmetic in your favour. The tool-schema filter needs no GPU at all: it uses BAAI/bge-small-en-v1.5, a 130 MB embedding model that runs on CPU. Install paritok[toolselect] on an ordinary VPS and you get the tool-block reduction for the price of a little RAM.
What happens if the compressor removes something the agent needed?
Nothing is removed. Compressed segments carry a [REF:id] tag and the model recovers the full text with read_original or expand_context. Filtered tool schemas are stubbed rather than deleted, and the model recovers one with gateway_search_tools. The real risk is quieter than a missing file: the model works from a lossy summary and never realises it should ask for the original. That is what the 86.5% quality-retained figure on SWE-bench Lite is measuring.
Why does my first request take fifteen seconds?
The embedding model behind the tool filter loads on the first request instead of at startup. The project documents a warm-up of 10 to 15 seconds, then roughly 15 ms per call after that. Send one throwaway request with curl after starting the proxy, and the first real agent turn will not stall.
Should I use the hosted GPU server instead of self-hosting?
It removes the GPU rent and the maintenance, priced at $0.30 per million tokens processed as of August 2026. It also sends your prompts and the files your agent reads to a third party before they reach your model provider. If you are self-hosting to keep code on infrastructure you control, that setting undoes the reason you started. Self-hosting keeps both the context and the provider API key on your own box.