Paritok Token Gateway: E Go Cut Agent Bill?
Paritok compresses file reads and tool output before API delivery. Project claims 74% fewer tokens; see how am works and when the savings break even.
Wetín Paritok dey do to request
Paritok na token gateway: proxy wey dey sit between your coding agent and model API, and e dey compress each request before e forward am. Your agent dey talk to http://127.0.0.1:8080 instead of the provider. The proxy dey shrink the tool schemas, file reads, tool output, and older turns, send the smaller payload go upstream, then hand the reply back unchanged.
Provider dey bill you based on wetin reach the provider, but the invoice depend on which tokens dem remove, their cache pricing, and how many turns the agent take. Na the whole idea be this. E different from claim say “your context go last longer”, and na why this tool dey interesting instead of just making things tidy.
The project still young. Dem open-sourced the gateway on 15 July 2026, and PyPI dates v1.3.14 to 6 October 2026 and v1.3.0 to 5 August. The weights and gateway code dey under Apache 2.0. The compression model na LoRA (low-rank adaptation) adapter for Qwen3-4B-Instruct-2507. Dem distill am from pool of 45,000 segments wey dem collect from real coding-agent trajectories. Out of these, 40,606 pass teacher validation and become the training examples.
Why this no be context trimming
Trimming dey delete content for good. When agent dey near its context limit and e drop the oldest turns, the file wey e read for turn 3 don disappear. If e need that file for turn 20, e go read the file again, so you go pay for those tokens another time. The saving na loan.
Paritok dey replace one segment with shorter form plus one tag, [REF:id], and e keep the full text for the proxy. Model fit recover segment by calling read_original or expand_context. This one change the way failure dey happen. Trimmer dey fail because e forget, and e no go tell you. Compressor dey fail by giving model excerpt wey lose some information, and model fit ask for the original when the excerpt no reach.
Filtered tool schemas still dey recoverable, but the default adaptive filter dey drop standard tools wey dem no select, and e dey drop MCP tools wey dem no select unless its MCP relevance signal fire. When that signal fire, unselected MCP tools dey turn to stubs. gateway_search_tools support recovery, and the gateway still dey detect missing-tool messages and inject matching tools before e retry. This matter because filter wey permanently hide one tool dey change wetin your agent fit do, and you go only discover am through task wey quietly go wrong.
The three levers, and which one free
First lever na the tool-schema filter. Every request carry the full tools array. For Claude Code turn wey no get MCP (model context protocol) servers attached, the project measure that block at about 29,000 tokens; when MCP dey attached, e report 44,000 to 65,000 tokens. The filter use BAAI/bge-small-en-v1.5, a 130 MB embedding model, to embed the user request and each tool description. E keep tools wey match, drop standard tools wey no select, and stub unselected MCP tools only when e MCP relevance signal fire. If not, e drop those MCP tools too. For the project's 29,000-token example, the block reduce to about 8,000 tokens. That embedding model dey run for CPU.
Second lever na content compression, and na this part use the 4B model, for GPU or through Ollama for CPU. E cut file reads, tool output, and history down to 25.7% of their original size. The model mainly select spans instead of rewriting dem, with small rewriting allowed for content like reasoning summaries and tool-call framing. The project's paper measure say 96.0% of emitted identifiers, paths, and numbers appear word-for-word inside the input for the teacher-generated training corpus; the trained model's held-out SWE-bench Lite output score 96.2% for the same audit. Na from here the 74% headline come. Read am well: 74% na the compression rate for content wey get compressed, no be the reduction for your bill.
Third lever na history summarization. By default, once context pass 80% of the configured window, e summarize older turns while four recent turns remain intact. This one make long session continue instead of hitting the limit.
You no need GPU if you dey self-host. Na the most useful sentence for this page. pip install "paritok[toolselect]" give you the tool filter for ordinary CPU VPS, with no per-token fee. Try am before you rent a card.
Wetín the project measure, and for whose harness
Di data wey dey behind dis chart
[
{
"label": "Paritok-4B-v1",
"compressed_to_pct": 25.7,
"quality_retained_pct": 86.5
},
{
"label": "gpt-4.1-mini",
"compressed_to_pct": 50.2,
"quality_retained_pct": 85.6
},
{
"label": "gpt-5",
"compressed_to_pct": 61.9,
"quality_retained_pct": 93.6
}
]Na the project own published figures be these, and dem measure am for the project own harness against SWE-bench Lite. Paritok-4B-v1 dey compress content to 25.7% of the original size, while e still retain 86.5% of the solve rate wey uncompressed version get. If you use gpt-5 as compressor, e retain more quality, 93.6%, but e compress only reach 61.9%. That one mean say you go pay frontier prices just to save frontier prices.
Read the quality column as e really be. Quality retained na compressor solve rate divided by uncompressed baseline solve rate. If e retain 86.5%, e mean say solves reduce by 13.5% net compared with that baseline. E no count the fraction of problems wey baseline solve but compressed run lose, because compressed runs fit still solve different problems. This figure measure single-shot understanding of compressed raw source, with recall disabled and production edit recovery enabled. E no measure the CPU tool-schema filter or the complete multi-turn gateway.
Di data wey dey behind dis chart
[
{
"label": "Turn 1",
"saved_pct": 25
},
{
"label": "Turn 5",
"saved_pct": 39
},
{
"label": "Turn 12",
"saved_pct": 57
},
{
"label": "Turn 20",
"saved_pct": 63
}
]End-to-end saving dey increase as session dey run, because history dey pile up and na history dem dey compress. The project report about 25% for one turn, 39% by turn 5, and 63% by turn 20. Dem measure only turns 1 to 5; the README label the later rows as projections. E also explain where the increase go stop: for 200,000 token budget, per-turn content saving dey flatten around 48,000 tokens, somewhere near turn 8 to 12, because once context full, history no dey grow again. The separate tool-filter saving add approximately 21,000 tokens per turn, so the projected combined plateau na approximately 69,000 tokens per turn. The "past 85%" figure wey people dey quote describe sessions wey context don full. Na the best-case result be that one, and the same README talk say saving dey approach ceiling of about 72% under default settings. So no plan based on am.
Paritok dey pay for 24GB GPU?
GPU no compulsory, and the model fit run for cards wey smaller than 24 GB. For one example rental calculation, use $0.44 per hour for 24 GB card. If you leave am running all month, na 730 hours, so e go cost $321. If you run am only during work hours, 8 hours every day for 22 days, na 176 hours, so e go cost $77.
Now convert the token reduction to dollar reduction. The reduction affect input tokens. Output tokens pass through the proxy unchanged, so the proxy no compress dem. Calculate fresh input, cache writes, and cache reads separately with your provider rates. Then compare the total API cost before and after. Even if input dey make up 80% of your total dollar cost, multiplying token reduction by 0.8 no prove how much your bill go reduce.
Di data wey dey behind dis chart
[
{
"label": "Always on (730 hours)",
"required_api_saving_usd": 321.2
},
{
"label": "Workday only (176 hours)",
"required_api_saving_usd": 77.44
}
]If you leave the card running, e don pay for itself once your measured monthly API saving pass $321.2, or $77.44 if you stop the instance outside work hours. 85% token reduction by itself no prove 68% bill saving, so e no fit prove monthly agent-spend threshold. Divide the rental cost by your measured bill-saving fraction to get that threshold.
Two things make this better than the chart dey show. The model no need 24 GB: the q4 build na about 2.5 GB and the f16 build na about 8 GB. So, smaller card, or GPU box wey you already dey run for another thing, go reduce every number for that chart. And stopping the instance when nobody dey code na the biggest lever here, because e cut the rent by about three quarters.
One thing make am worse. The compression pass na real work. Every token wey the 4B model compress na token wey e must read, and every span wey e keep na one wey e must write back out. This add latency to every agent turn. If you rent the card by the hour, that cost show as waiting, not as line for invoice. So e easy to miss until you feel am.
If you dey compare rented GPU hours with API tokens generally, the break-even between GPU VPS and API tokens dey use the same arithmetic for the inference itself.
Paritok gateway VPS for run
PyPI package metadata Python 3.10 or newer accept dey, although README badge talk say 3.11+. Ubuntu 24.04 dey ship Python 3.12, so plain VPS image dey enough for CPU tool filter.
sudo apt update && sudo apt install -y python3-venv curl
python3 -m venv /opt/paritok/venv
source /opt/paritok/venv/bin/activate
pip install "paritok[proxy]==1.3.14"
pip install "paritok[toolselect]==1.3.14"Pin the version. PyPI date v1.3.0 as 5 August 2026 and v1.3.14 as 6 October 2026, and project wey dey move that fast fit rename things between releases: v1.3.0 itself rename expand_context to read_original for its API. Bare pip install paritok, or git clone of main, go give you different gateway next week and leave no record of which one produce the numbers wey you measure.
Default backend na Ollama. Pull the model, then give am the short name wey proxy dey look for.
ollama pull paritok/paritok-4b-v1
ollama cp paritok/paritok-4b-v1 paritok-4b-v1Because local model dey write out the spans wey e keep for every segment wey e compress, compression pass wey run too long go show as stalled agent turn, and Ollama num_predict cap for output length na the knob wey dey limit am.
Write paritok.yaml next to am. use_gpu_server: false na wetin keep the compression for your own hardware.
use_gpu_server: false
local_model:
base_url: http://localhost:11434/v1paritok proxy --port 8080 --config-file paritok.yamlparitok up na shortcut to pull the model if e no dey and start proxy for port 8080; Ollama must don install already. Check the proxy before you point agent give am.
curl http://127.0.0.1:8080/health
curl http://127.0.0.1:8080/stats/health dey return small JSON object wey contain "status":"ok" and version string. /stats dey return compression totals and proxy own estimate of wetin e save. Treat that estimate as proxy wey dey grade its own work, and confirm am against your provider usage page.
For throughput instead of convenience, vLLM dey serve the adapter on top of the base model.
vllm serve Qwen/Qwen3-4B-Instruct-2507 \
--enable-lora \
--lora-modules paritok-4b-v1=paritok/paritok-4b-v1 \
--max-lora-rank 32 \
--port 8000--max-lora-rank 32 flag dey inside project own command because adapter na rank 32. Then set local_model.base_url to http://localhost:8000/v1 for paritok.yaml so proxy go talk to vLLM instead of Ollama.
Ollama quick to set up. vLLM handle concurrent requests much better, and this begin matter as soon as more than one agent share the box. The practical difference between Ollama and vLLM na wetin go decide this one for you.
Point agent to proxy with the base URL environment variables.
export ANTHROPIC_BASE_URL=http://127.0.0.1:8080
export OPENAI_BASE_URL=http://127.0.0.1:8080Codex CLI no dey obey OPENAI_BASE_URL, so paritok up write ~/.codex/config.toml for you when codex.enabled: true dey set inside paritok.yaml. If you export the variable alone, Codex go still talk straight to provider, and sign of this na /stats counter wey no ever move while you dey work.
Keep listener for 127.0.0.1, never for 0.0.0.0. Proxy dey forward the credentials wey each request supply upstream, and its unauthenticated /stats endpoint dey expose samples of original and compressed content, so listener wey face internet go expose private context and local compression resources. Reach am from laptop through SSH tunnel or VPN instead of opening the port.
Run am under systemd so e go survive reboot. Adjust the paths to match your install. Create paritok user wey unit go run as, and give am the install directory:
sudo useradd --system --home-dir /opt/paritok --shell /usr/sbin/nologin paritok
sudo chown -R paritok:paritok /opt/paritok[Unit]
Description=Paritok compression proxy
After=network-online.target
[Service]
User=paritok
WorkingDirectory=/opt/paritok
ExecStart=/opt/paritok/venv/bin/paritok proxy --port 8080 --config-file /opt/paritok/paritok.yaml
Restart=on-failure
[Install]
WantedBy=multi-user.targetEnable am with sudo systemctl enable --now paritok, then curl /health again. Unit wey start and exit immediately usually mean say config file path wrong, and journalctl -u paritok -n 50 go print the reason.
Wetin the hosted option go cost you
The project still dey offer the compression as a service. Set use_gpu_server: true with an API key from paritok.com, and the 4B model go run for their hardware. As of 2 November 2026, dem dey charge $0.30 per million file tokens wey e compress, using prepaid credit, while tool-schema filtering dey free. New accounts get $5 free credit, and card no dey required. The launch period when the hosted service dey free end on 1 October 2026, so na paid service now. This one remove the work of hosting the compression model, but you still dey run the gateway.
E also mean say your prompts and the files wey your agent dey read go leave your machine and reach third party before dem reach your model provider. Self-hosting dey exist to avoid exactly this hop. Decide which one you want optimize for before you set that flag, because changing the flag na one-line work, but the consequence no be.
How to measure your own before and after
The published numbers na the project own numbers, from the project harness, for SWE-bench Lite. Your repository no be SWE-bench Lite. Measure your own results.
- Run one normal week with no proxy for the path. Record fresh input, cache-write, cache-read and output tokens as separate lines from your provider usage page, no be one dollar total.
- Run the next week with the proxy in front, while you dey do the same kind work.
- Compare the input categories with their actual prices, and include any change for output cost. The proxy no compress output, but changes for agent behaviour and turn count still affect how much output you buy.
- Count the tasks wey you need redo. Na the quality side of the trade, and no dashboard anywhere dey report am.
- Add the GPU hours to week two before you compare the totals.
To split input from output matter because dem get very different prices, and compressor dey affect only one of dem. As of 2 November 2026, Claude Sonnet 4.6 cost $3 per million input tokens and $15 per million output tokens, while prompt-cache read na 10% of the input rate, $0.30 per million. The newer Claude Sonnet 5.5 cost $2 and $10, and e cache reads na 5% of the input rate, $0.10 per million. The difference between input and output token cost na wetin decide whether input-side compressor go benefit you at all. Where Claude Code tokens actually dey go go show you which part of your context big enough to make compression worth am.
Prompt caching dey make the tool-filter calculation especially complicated. The tool block dey for the front of the request, so after the first turn e normally become cache hit; for Sonnet 4.6, that one cost 10% of the input price. If you cut 21,000 tokens from cached block, you save 21,000 at $0.30 per million, about $0.006 per turn, instead of the $0.063 wey uncached rate go suggest. The project keep the filtered block frozen for the session so the cached prefix no change. Filter wey dey pick tools again every turn go invalidate that prefix, and e fit cost pass wetin e save.
Wetin never get independent confirmation
Every performance number wey dey above come from the project itself. Independent synthetic-corpus compression check dey available, but e clearly no test SWE-bench task success; the project's SWE-bench Lite solve-quality figures still never get independent confirmation. Since dem open-source the gateway for July 2026, the code never get much operational history too. The party wey benefit if the compression rate and quality-retained figure look good na the same party wey measure both of dem. That no mean say dem wrong. E mean say dem never get confirmation, so you suppose treat dem differently from number wey you produce by yourself.
One documented behaviour good make you know before you blame your setup. The embedding model wey the tool filter dey use loads for the first request, instead of when the service starts. So the project document warm-up of 10 to 15 seconds, then around 15 ms per call afterwards. Send one throwaway request after the proxy starts, and your first real agent turn no go look like e hang.
Four things you fit settle by yourself in one afternoon: whether the proxy starts and stays up, whether /stats dey move as you work, whether your provider's input-token line actually dey reduce, and whether the agent still dey finish the work. Those ones go decide am for your setup much better than any published benchmark.
About where this one fit beside your other tooling: self-hosted LiteLLM gateway dey route and meter requests without changing their contents, so both tools dey solve different problems and you fit chain dem, with Paritok sitting nearest to the agent. If the real goal na smaller bill instead of this specific tool, the wider set of cost controls for an agent on a VPS get several changes wey cost nothing to try first.
FAQ
Paritok dey reduce my API bill, or na only my context usage e dey reduce?
E dey make the request smaller before e reach provider. The money wey you save depend on the price of the tokens wey dem remove and the number of turns wey the agent make after that. The reduction size smaller than the headline dey suggest. The 74% figure na the compression rate for the content wey dem compress. From beginning to end, the project report about 25% saving for one turn, and project 63% saving by turn 20 for input tokens. Output tokens pass through untouched, but the amount wey the agent generate still depend on agent behaviour.
How much GPU I need to self-host the compression model?
GPU no compulsory: Ollama fit run the model for CPU too. The q4 build na about 2.5 GB, while the f16 build na about 8 GB. So the model fit enter 24 GB card with plenty space still remaining. Smaller card fit work, and e go make the break-even calculation favour you. The tool-schema filter no need GPU at all: e dey use BAAI/bge-small-en-v1.5, wey na 130 MB embedding model wey dey run for CPU. Install paritok[toolselect] for ordinary VPS, and you go get the tool-block reduction for the cost of small RAM.
Wetin go happen if the compressor remove something wey the agent need?
Nothing go lost permanently. The model dey delete spans from wetin e forward, but compressed segments carry [REF:id] tag, and the model fit recover the full text with read_original or expand_context. The default tool filter dey drop standard tools wey dem no select, and e dey drop MCP tools wey dem no select unless the MCP relevance signal trigger. If e trigger, e go create stubs for those MCP tools. Tool recovery dey use gateway_search_tools and detection of missing-tool messages. The real risk quieter than missing file: the model dey work from lossy excerpt and e no realise say e suppose ask for the original. The 86.5% quality-retained figure for SWE-bench Lite measure single-shot understanding of compressed raw source with recall disabled and production edit recovery enabled. E no measure the CPU tool-schema filter or the complete multi-turn gateway.
Why my first request dey take fifteen seconds?
The embedding model behind the tool filter dey load for the first request instead of when the service start. The project document warm-up of 10 to 15 seconds, then about 15 ms for each call after that. Send one throwaway request with curl after you start the proxy, so the first real agent turn no go stall.
I suppose use the hosted GPU server instead of self-hosting?
E remove the need to host the compression model and pay for GPU, but you still dey maintain the gateway. As of 2 November 2026, e cost $0.30 per million file tokens wey dem compress. Dem dey pay am from prepaid credit, and new accounts get $5 free credit. The free launch period end on 1 October 2026. The service also dey send your prompts and the files wey your agent read to third party before dem reach your model provider. If you self-host to keep code for infrastructure wey you control, this setting dey cancel the reason wey make you start. Self-hosting keep compression for your own machine; compressed requests and provider credentials still go upstream to your model provider.