Paritok Token Gateway: E Fit Cut Agent Bills?

Paritok dey do to request

Paritok na token gateway: proxy wey dey between your coding agent and the model API, and e dey compress each request before e forward am. Your agent dey talk to http://127.0.0.1:8080 instead of the provider. The proxy dey rewrite the tool schemas, file reads, tool output, and older turns, send the smaller payload upstream, then hand the reply back unchanged.

Provider dey bill you based on wetin reach provider, so smaller payload mean smaller invoice. Na the whole idea be that. E different from the claim say “your context go last longer”, and na why this tool dey interesting instead of just making things tidy.

The project still young. Its first public tags get July 2026 dates, and the current tag na v1.3.0, dated 5 August 2026. The weights and gateway code dey under Apache 2.0. The compression model na LoRA (low-rank adaptation) adapter for Qwen3-4B-Instruct-2507, trained with 45,000 teacher-distilled samples wey come from real coding-agent trajectories.

Why dis one no be context trimming

Trimming dey delete things. When agent dey near its context limit and e drop the oldest turns, the file wey e read for turn 3 don disappear. If e need that file for turn 20, e go read the file again, so you go pay for those tokens another time. The saving na loan.

Paritok dey replace one segment with shorter form plus one tag, [REF:id], and e keep the full text for the proxy. Model fit recover the segment by calling read_original or expand_context. This one change the failure mode. Trimmer dey fail because e forget, and e no go ever tell you. Compressor dey fail because e hand model one lossy summary, and model fit ask for the original when the summary no enough.

The tool filter dey work the same way. Filtered tool schemas dey become stubs instead of dem removing dem, and model fit recover one by calling gateway_search_tools. This matter because filter wey hide tool permanently go change wetin your agent fit do, and you go only discover am through task wey quietly go wrong.

Lever three, and which one no cost

Lever one na tool-schema filter. Every request carry the complete tools array. For one Claude Code turn wey get some MCP (model context protocol) servers attached, project measure that block at around 29,000 tokens. The filter embed user request and each tool description with BAAI/bge-small-en-v1.5, a 130 MB embedding model, keep the tools wey match, then stub the remaining ones. The block reduce reach around 8,000 tokens. That embedding model dey run for CPU.

Lever two na content compression, and na this part need the 4B model for GPU. File reads, tool output, and history dey rewrite to 25.7% of their original size. Na there the 74% headline come from. Read am well: 74% na the compression rate for the content wey dem compress, no be the reduction for your bill.

Lever three na history summarization. Once the context budget full, dem summarize turns wey pass the recent window so long session fit continue instead of reaching the limit.

Na lever two alone need GPU. Na the most useful sentence for this page. pip install "paritok[toolselect]" give you the tool filter for ordinary CPU VPS, and na the half of the product wey no cost you monthly. Try am before you rent a card.

Wetín the project measure, and for whose harness

[
  {
    "label": "Paritok-4B-v1",
    "compressed_to_pct": 25.7,
    "quality_retained_pct": 86.5
  },
  {
    "label": "gpt-4.1-mini",
    "compressed_to_pct": 50.2,
    "quality_retained_pct": 85.6
  },
  {
    "label": "gpt-5",
    "compressed_to_pct": 61.9,
    "quality_retained_pct": 93.6
  }
]

Na the project own published figures be these, and e measure dem with im own harness against SWE-bench Lite. Paritok-4B-v1 compress content reach 25.7% of the original size, while e still keep 86.5% of the solve rate wey no compress. If you use gpt-5 as the compressor, e keep more quality, 93.6%, but e only compress reach 61.9%, and you go dey pay frontier prices just to save frontier prices.

Read the quality column as e be. To keep 86.5% of the solve rate mean say the compressed runs fail problems wey the uncompressed runs solve, almost one solve out of seven. For benchmark, na just number for table. For your repository, na task wey you go run twice.

[
  {
    "label": "Turn 1",
    "saved_pct": 25
  },
  {
    "label": "Turn 5",
    "saved_pct": 39
  },
  {
    "label": "Turn 12",
    "saved_pct": 57
  },
  {
    "label": "Turn 20",
    "saved_pct": 63
  }
]

End-to-end saving dey increase as session dey run, because history dey accumulate and na the history dem dey compress. The project report about 25% for one turn, 39% by turn 5, and 63% by turn 20. E still explain where the increase stop: with 200,000 token budget, the absolute saving flatten around 48,000 tokens per turn, somewhere near turn 8 to 12, because once context full, history no dey grow again. The "past 85%" figure wey people dey quote describe sessions wey context don full. Na the best case, so no plan based on am.

24GB GPU go pay for Paritok?

24 GB card na the normal rental unit for model wey reach this size. As of 7 August 2026, the median published on-demand rate for RTX 4090 wey get 24 GB na $0.44 per hour, while the cheapest listings dey around $0.20. Make we use $0.44. If e dey run the whole month, na 730 hours, so e go cost $321. If you run am only during working hours, 8 hours per day for 22 days, na 176 hours, so e go cost $77.

Now convert the token reduction to dollar reduction. The reduction apply to input tokens. Output tokens pass through the proxy without change, so dem no change at all. Assume say input tokens na 80% of your total bill, as e dey normal for coding agent, and compare this assumption with your own bill. Your dollar saving go be the token reduction multiplied by 0.8.

[
  {
    "label": "Turn 5 (39% saved)",
    "bill_always_on_usd": "1,030",
    "bill_workday_only_usd": 248
  },
  {
    "label": "Turn 20 (63% saved)",
    "bill_always_on_usd": 637,
    "bill_workday_only_usd": 154
  },
  {
    "label": "Saturated (85% saved)",
    "bill_always_on_usd": 472,
    "bill_workday_only_usd": 114
  }
]

For the saturated-session figure of 85%, you keep 68% of the bill. So, GPU card wey dey run always go pay for itself once your monthly agent spend pass about $472, or about $114 if you stop the instance outside working hours. For the turn-20 figure of 63%, the values become $637 and $154. For the turn-5 figure of 39%, wey na wetin short sessions really look like, you need about $1,030 per month before renting the card make sense at all.

Two things make this better than wetin the table show. The model no need 24 GB: the q4 build na about 2.5 GB and the bf16 build na about 8 GB. So, smaller card, or GPU box wey you already dey run for another work, go reduce every number for that chart. Also, stopping the instance when nobody dey code na the biggest lever here, because e go cut the rent by about three quarters.

One thing make am worse. The compression pass na real work. Every token wey the 4B model compresses na token wey e must first read, then write. This add latency to every agent turn. For card wey you rent by the hour, this cost go show as waiting, not as line for invoice. So, e easy to miss until you start to feel am.

If you dey compare rented GPU hours with API tokens generally, the break-even between a GPU VPS and API tokens dey use the same arithmetic for the inference itself.

Paritok gateway dey run for a VPS

Python 3.10 or newer na requirement. Ubuntu 24.04 dey ship Python 3.12, so plain VPS image dey enough for the CPU-only part.

sudo apt update && sudo apt install -y python3-venv curl
python3 -m venv /opt/paritok/venv
source /opt/paritok/venv/bin/activate
pip install "paritok[proxy]==1.3.0"
pip install "paritok[toolselect]==1.3.0"

Pin the version. Repository tag v1.2.8 on 29 July 2026 and v1.3.0 on 5 August 2026, and project wey dey move at that speed fit rename config keys between releases. Bare pip install paritok, or git clone of main, go give you different gateway next week and no record go remain of which one produce the numbers wey you measure.

Default backend na Ollama. Pull the model, then give am the short name wey proxy dey look for.

ollama pull paritok/paritok-4b-v1
ollama cp paritok/paritok-4b-v1 paritok-4b-v1

Write paritok.yaml beside am. use_gpu_server: false na wetin keep the compression for your own hardware.

use_gpu_server: false
local_model:
  base_url: http://localhost:11434

paritok proxy --port 8080 --config-file paritok.yaml

paritok up na shortcut for everything above: e go pull the model if e no dey, then start proxy for port 8080. Check the proxy before you point agent go am.

curl http://127.0.0.1:8080/health
curl http://127.0.0.1:8080/stats

/health dey return small JSON object wey contain "status":"ok" and version string. /stats dey return compression totals and proxy own estimate of wetin e save. Treat that estimate as proxy wey dey grade its own work, then confirm am against your provider usage page.

For throughput instead of convenience, vLLM dey serve the adapter on top of the base model.

vllm serve Qwen/Qwen3-4B-Instruct-2507 \
  --enable-lora \
  --lora-modules paritok-4b-v1=paritok/paritok-4b-v1 \
  --port 8000

Ollama dey quicker to set up. vLLM dey handle concurrent requests much better, and this start to matter as soon as more than one agent share the box. The practical difference between Ollama and vLLM na wetin go decide this one for you.

Point the agent to the proxy with the base URL environment variables.

export ANTHROPIC_BASE_URL=http://127.0.0.1:8080
export OPENAI_BASE_URL=http://127.0.0.1:8080

Codex CLI no dey use OPENAI_BASE_URL, so project dey write ~/.codex/config.toml for you when codex.enabled: true dey set inside paritok.yaml. If you export the variable alone, Codex go talk directly to provider, and sign of this na /stats counter wey no ever move while you dey work.

Keep the listener on 127.0.0.1, never on 0.0.0.0. Proxy dey forward your provider API key upstream, so proxy wey internet fit reach na open relay for that key: anybody wey find the port fit spend your money without ever seeing the key itself. Reach am from laptop through SSH tunnel or VPN instead of opening the port.

Run am under systemd so e go survive reboot. Adjust the paths to match your install.

[Unit]
Description=Paritok compression proxy
After=network-online.target

[Service]
User=paritok
WorkingDirectory=/opt/paritok
ExecStart=/opt/paritok/venv/bin/paritok proxy --port 8080 --config-file /opt/paritok/paritok.yaml
Restart=on-failure

[Install]
WantedBy=multi-user.target

Enable am with sudo systemctl enable --now paritok, then curl /health again. Unit wey start and immediately exit usually mean say config file path no correct, and journalctl -u paritok -n 50 go print the reason.

The hosted option, and wetin e go cost you

The project dey also offer the compression as a service. Set use_gpu_server: true with an API key, and the 4B model go run for their hardware. The price na $0.30 for every million tokens wey e process, and according to their own documentation, e free until the end of August 2026. This removes GPU rent and all the operations work wey we describe above.

But e also mean say your prompts and the files wey your agent dey read go comot from your machine reach third party before dem reach your model provider. Self-hosting dey to avoid exactly that extra hop. Decide which one you dey optimize for before you set that flag, because na one-line change the flag be, but the consequence no be.

How to measure your own before and after

The published numbers na the project own numbers, from the project harness, for SWE-bench Lite. Your repository no be SWE-bench Lite. Measure your own results.

• Run one normal week with no proxy for the request path. Record input tokens, cache-read tokens, and output tokens as separate lines from your provider usage page. No use one dollar total.
• Run the next week with the proxy in front, while you dey do the same kind work.
• Compare the input and cache-read lines. Output suppose remain roughly flat, because nothing dey compress am. If output change plenty, something else don change apart from the proxy.
• Count the tasks wey you need redo. Na the quality side of the trade-off, and no dashboard anywhere dey report am.
• Add the GPU hours to week two before you compare the totals.

E important to separate input from output because dem get very different prices, and compressor only dey affect one of dem. As of August 2026, Claude Sonnet 4.6 cost $3 per million input tokens and $15 per million output tokens. A prompt-cache read na 10% of the input rate, or $0.30 per million. The difference between input and output token cost na wetin decide whether input-side compressor go benefit you. Where Claude Code tokens actually go go show you which part of your context big enough to make compression worth doing.

Prompt caching dey make the tool-filter calculation more complicated, especially. The tool block dey for the front of the request, so after the first turn e normally become cache hit at 10% of the input price. If you cut 21,000 tokens from cached block, you save 21,000 tokens at $0.30 per million, about $0.006 per turn, instead of the $0.063 wey uncached rate imply. The project dey keep the filtered block frozen for the session, so the cached prefix no dey change. If filter dey select tools again every turn, e go invalidate that prefix and cost more than wetin e save.

Wetin still never verify

Every performance number for up here come from the project itself. Nobody independently reproduce the SWE-bench Lite results, and since the first tags get July 2026 date, the code never get enough operational history yet. The compression rate and the quality-retained figure both come from the party wey benefit if dem look good. That no mean say dem wrong. E mean say dem never confirm, and you suppose treat dem differently from number wey you produce yourself.

One documented behaviour dey important to know before you blame your setup. The embedding model wey the tool filter use dey load for the first request, instead of when the tool start. So the project document warm-up of 10 to 15 seconds, then around 15 ms for each call after that. Send one throwaway request after the proxy start, and your first real agent turn no go look like e hang.

Four things you fit settle yourself for one afternoon: whether the proxy start and remain up, whether /stats dey move as you work, whether your provider input-token line actually dey reduce, and whether the agent still finish the work. Those ones go decide the matter for your setup much better than any published benchmark.

About how this fit alongside your other tooling: a self-hosted LiteLLM gateway routes and meters requests without changing their contents, so both tools solve different problems and fit chain together, with Paritok sitting closest to the agent. If the real goal na smaller bill instead of this particular tool, the wider set of cost controls for an agent on a VPS get several changes wey cost nothing to try first.

FAQ

Paritok dey reduce my API bill, or na only my context usage e dey reduce?

E dey reduce the bill, because the proxy dey rewrite request before e reach provider, and provider dey charge for wetin e receive. The size of the reduction smaller than the headline suggest. The 74% figure na compression rate for the content wey dem compress. From start to finish, the project report about 25% for one turn and 63% by turn 20, and na input tokens only dey change. Output tokens pass through untouched.

How much GPU I need to self-host the compression model?

The q4 build na about 2.5 GB and the bf16 build na about 8 GB, so the model fit inside 24 GB card with plenty space still dey. Smaller card fit work, and e go make the break-even calculation favour you. The tool-schema filter no need GPU at all: e dey use BAAI/bge-small-en-v1.5, na 130 MB embedding model wey fit run for CPU. Install paritok[toolselect] for ordinary VPS and you go get tool-block reduction for the cost of small RAM.

Wetin go happen if the compressor remove something wey the agent need?

Nothing dey removed. Compressed segments carry [REF:id] tag, and the model recover the full text with read_original or expand_context. Filtered tool schemas dey become stubs instead of deletion, and the model recover one with gateway_search_tools. The real risk quieter than missing file: the model dey work from lossy summary and e no realise say e suppose ask for the original. Na wetin the 86.5% quality-retained figure for SWE-bench Lite dey measure.

Why my first request dey take fifteen seconds?

The embedding model behind the tool filter dey load for the first request instead of during startup. The project document warm-up of 10 to 15 seconds, then about 15 ms per call after that. Send one throwaway request with curl after you start the proxy, and the first real agent turn no go stall.

I suppose use the hosted GPU server instead of self-hosting?

E remove GPU rent and maintenance, with price of $0.30 per million tokens processed as of August 2026. E also send your prompts and the files wey your agent dey read go third party before dem reach your model provider. If you dey self-host to keep code for infrastructure wey you control, that setting undo the reason wey make you start. Self-hosting keep both the context and the provider API key for your own box.