By GPU VPS or API tokens: when e cost less?
GPU VPS only beat per-token API billing after you pass a monthly token volume. See the formula and exact break-even count for each model type.
Where break-even really dey happen
GPU VPS go beat per-token API billing for one point: when you divide the fixed monthly rent by the output tokens wey you actually generate that month, and the result fall below wetin API dey charge for the same tokens. The rent no dey change. API bill dey change with every request. So the answer always na monthly volume, never simple yes or no.
Use mid-range GPU VPS wey cost $0.50 per hour. That one na $365 for 730-hour month. Against frontier model API, you go reach break-even at 24.3 million output tokens per month. Against small commercial model, na 73 million. Against hosted open-weight model of the same size, you no go ever reach break-even, because one card no fit generate enough tokens for one month to reach the crossing point.
Published break-even studies no answer this question. Two studies from earlier for 2026 put the crossover near 72% sustained utilisation for H200, and between 22% and 48% duty cycle for MI300X. Both compare against the same vendor own serverless product, and both use accelerator prices wey cost more per hour than wetin most readers here spend for one month. The arithmetic remain the same. Wetin follow go calculate am again for one card, one open model for the 7B to 30B range, and ordinary metered API billing.
Every number below na input, no be result. Replace all of dem with your own numbers.
Formula wey go make you put your own numbers
cost_per_million = (hourly_rate * 1000000) / (tokens_per_second * 3600 * duty_cycle)
breakeven_millions_per_month = (hourly_rate * hours_per_month) / api_price_per_million
capacity_millions_per_month = (tokens_per_second * 3600 * hours_per_month) / 1000000
required_duty_cycle = breakeven_millions_per_month / capacity_millions_per_monthInputs dey four, and you fit measure or find all four.
hourly_ratena wetin GPU VPS dey cost per hour, including hours wey e dey idle. If na monthly payment you dey make, divide monthly price by 730.tokens_per_secondna the total output rate wey your server dey maintain with your real concurrency. E no be the single-stream number wey vendor chart show.duty_cyclena the part of the month wey GPU dey spend generating tokens. If you rent one machine for the whole month but use am for two hours every day, e go be 8.3%.api_price_per_millionna the metered price wey you dey compare against, for output tokens.
The example dey price output tokens for both sides, because output dey make up most of the bill for chat and agent work. If your prompts long, add input for both sides. For API side, na separate line for invoice. For your own card, prefill dey use GPU time, so e don already show as lower measured tokens_per_second.
How to measure tokens per second before you trust the arithmetic
Everything wey dey above depend on one number we measure. If you get am wrong by factor of three, answer go wrong by factor of three too. Measure am for the card wey you dey rent, with the model and quantisation wey you really go run.
Ollama dey give you the single-stream figure with one command:
ollama run qwen3:8b --verbose "Write 400 words about disk latency."--verbose dey print timing block after the answer. The line wey matter na eval rate, for tokens per second, wey dey count generation only. prompt eval rate na prefill speed and e normally much higher. Your numbers go different from these:
eval count: 412 token(s)
eval duration: 9.612s
eval rate: 42.86 tokens/sSingle stream no be the correct number for cost model, because e dey measure one request at a time for card wey fit serve many requests together. For aggregate figure, serve the model with vLLM and read the throughput wey the server report about itself:
pip install vllm
vllm serve Qwen/Qwen3-8B --max-model-len 8192As requests dey in flight, server dey log status line for every reporting interval. The exact fields dey change between vLLM releases, so read your own output instead of my own:
Avg prompt throughput: 812.4 tokens/s, Avg generation throughput: 396.1 tokens/s, Running: 16 reqs, Waiting: 0 reqs, GPU KV cache usage: 21.7%Avg generation throughput na the number wey formula need. E dey increase as you add concurrent requests until KV cache (key-value cache, the attention state for each request wey vLLM dey keep for VRAM) full, then e stop increasing. If you pass that point, requests go queue instead of becoming faster, and you go see am as rising Waiting count. vLLM also ship load generator, vllm bench serve. Its flags dey change between versions, so run vllm bench serve --help for the version wey you install instead of copying command from blog post.
Monitor the card while the test dey run:
nvidia-smi --query-gpu=utilization.gpu,memory.used,power.draw --format=csv -l 5If utilization.gpu dey near 100% during generation, throughput na the limit and the number wey you measure na the real ceiling. If e stay low, something else be the limit: concurrent requests too few, client slow, or model no fit inside VRAM and e dey partly offload to system RAM. Ollama and vLLM dey make very different trade-offs here, and the gap between dem for the same card wide enough to move your break-even by factor of several.
How the bill go look across one month
The worked example na one 24 GB GPU VPS for $0.50 per hour, with one 8B open model wey vLLM dey serve. E dey produce 400 output tokens per second altogether, with 16 requests running at the same time. Rent no change whether you use the card or not. For that rate, capacity na 1,051 million output tokens per month. Na wetin the card go produce if e no ever stop.
The data behind this chart
[
{
"output_tokens_millions": 5,
"gpu_vps_usd": 365,
"open_api_usd": 1,
"small_api_usd": 25,
"frontier_api_usd": 75
},
{
"output_tokens_millions": 10,
"gpu_vps_usd": 365,
"open_api_usd": 2,
"small_api_usd": 50,
"frontier_api_usd": 150
},
{
"output_tokens_millions": 25,
"gpu_vps_usd": 365,
"open_api_usd": 5,
"small_api_usd": 125,
"frontier_api_usd": 375
},
{
"output_tokens_millions": 50,
"gpu_vps_usd": 365,
"open_api_usd": 10,
"small_api_usd": 250,
"frontier_api_usd": 750
},
{
"output_tokens_millions": 100,
"gpu_vps_usd": 365,
"open_api_usd": 20,
"small_api_usd": 500,
"frontier_api_usd": 1500
},
{
"output_tokens_millions": 250,
"gpu_vps_usd": 365,
"open_api_usd": 50,
"small_api_usd": 1250,
"frontier_api_usd": 3750
},
{
"output_tokens_millions": 500,
"gpu_vps_usd": 365,
"open_api_usd": 100,
"small_api_usd": 2500,
"frontier_api_usd": 7500
},
{
"output_tokens_millions": 1000,
"gpu_vps_usd": 365,
"open_api_usd": 200,
"small_api_usd": 5000,
"frontier_api_usd": 15000
}
]The GPU line flat for 365 dollars because rent no care wetin you dey do with the card. Every API line na straight line wey start from zero. Each pair cross only once.
For 25 million output tokens per month, the frontier API bill 375 dollars, so the two sides dey within ten dollars of each other. For 50 million, the small commercial model bill 250 dollars and still remain the cheaper option. For 1000 million output tokens, wey need the card busy for 95% of the month, the hosted open-weight API bill 200 dollars against the same rent. The card lose by almost two times for the exact volume wey make am work hardest.
That last result dey surprise people, but e no happen by accident. Hosted open-weight endpoint na GPU fleet wey dey run with high utilisation, so the price dey close to the cost of one saturated card. You no fit beat saturated fleet by renting one card and running am below full load. Wetin you fit beat na frontier pricing, because capability, not silicon time, dey set the price.
Cost per million output tokens for each duty cycle
Volume and duty cycle na the same matter wey you dey look from two sides. Rent dey buy hours. Idle hours no dey produce anything, but dem still dey cost money.
The data behind this chart
[
{
"label": "100% duty",
"self_host_usd_per_million": "0.35",
"open_api_usd_per_million": "0.20",
"small_api_usd_per_million": "5.00",
"frontier_api_usd_per_million": "15.00"
},
{
"label": "50% duty",
"self_host_usd_per_million": "0.69",
"open_api_usd_per_million": "0.20",
"small_api_usd_per_million": "5.00",
"frontier_api_usd_per_million": "15.00"
},
{
"label": "25% duty",
"self_host_usd_per_million": "1.39",
"open_api_usd_per_million": "0.20",
"small_api_usd_per_million": "5.00",
"frontier_api_usd_per_million": "15.00"
},
{
"label": "10% duty",
"self_host_usd_per_million": "3.47",
"open_api_usd_per_million": "0.20",
"small_api_usd_per_million": "5.00",
"frontier_api_usd_per_million": "15.00"
},
{
"label": "5% duty",
"self_host_usd_per_million": "6.94",
"open_api_usd_per_million": "0.20",
"small_api_usd_per_million": "5.00",
"frontier_api_usd_per_million": "15.00"
},
{
"label": "2% duty",
"self_host_usd_per_million": "17.36",
"open_api_usd_per_million": "0.20",
"small_api_usd_per_million": "5.00",
"frontier_api_usd_per_million": "15.00"
}
]The three API columns na typical published list prices as of August 2026: 0.20 dollars per million output tokens for hosted 8B open-weight model, 5.00 dollars for small commercial model, and 15.00 dollars for frontier model. Dem na examples. Check today's price page before you decide anything. If your comparison na against flat monthly plan instead of metered tokens, the subscription calculation dey work differently and the crossing point go move again.
Run the card flat out and one million output tokens go cost 0.35 dollars, and this one genuinely cheap. For 10% duty cycle, the same one million go cost 3.47 dollars. For 2% duty cycle, e go cost 17.36 dollars. This one no dey the same range as the 0.20 dollars wey hosted open model dey charge for the same output.
Below roughly 10% duty cycle, renting GPU na the expensive choice. You dey pay 3.47 dollars per million tokens for output wey dey sell for 0.20 dollars. Wetin you dey buy with the difference na privacy and bill wey no dey change. Those benefits fit really worth money. But dem no be price advantage, so no record dem as one.
The break-even volume for each API tier
The data behind this chart
[
{
"label": "Hosted open 8B API",
"breakeven_tokens_millions": 1825,
"required_duty_pct": 174
},
{
"label": "Small commercial model",
"breakeven_tokens_millions": 73,
"required_duty_pct": 6.9
},
{
"label": "Frontier model",
"breakeven_tokens_millions": 24.3,
"required_duty_pct": 2.3
}
]Against the frontier tier, you need 24.3 million output tokens every month. Na na 2.3% of wetin the card fit do. This threshold low. Small team wey dey run coding agents throughout workday fit pass am.
Against the small commercial tier, you need 73 million tokens every month, or 6.9% duty cycle. Against the hosted open-weight tier, the required duty cycle na 174%. Anything pass 100% no possible by definition: the card go need run more hours than the month get. One mid-range card for this hourly rate no fit win that comparison, so na only cheaper card, faster card, or reason wey no be price fit change the result.
Wetin formula dey hide
Formula dey price GPU hours and tokens. Several real costs dey outside am.
Cold starts. An 8B model wey get 16-bit weights na about 16 GB, and to load am from local disk enter VRAM fit take tens of seconds. If you stop the box between uses to save rent, you go pay that wait for the first request every time. If you leave am running to avoid the wait, your duty cycle go collapse, and cost per token go increase. Na this trade-off be the main reason serverless inference dey exist.
Storage and download. Weights big well-well. An 8B model for 16-bit na around 16 GB, a 30B model wey you quantise to 4-bit na around 18 GB, and 30B model for 16-bit no fit enter 24 GB card at all. The limit dey become clear quickly for the top end, where to run trillion-parameter open model like Kimi K3 mean say the weights alone don pass any single card wey you fit rent by the hour. You go pay for that disk every month, and you go pay with time for every rebuild. Run du -sh ~/.cache/huggingface/hub after one week of experiments. E dey grow pass wetin you expect, because every quantisation wey you try once still dey there.
Your own time. Driver and CUDA versions, out-of-memory errors for context length wey work yesterday, and model update wey change chat template. None of dem dey show for cost-per-token figure, but na your evenings dem dey take pay for. If you never size one of these boxes before, wetin GPU VPS actually give you worth reading before you commit to one month of rent.
The quality gap. Na this be the biggest hidden cost and the hardest one to price. An 8B open model no be frontier model. If e need three attempts where frontier model need one, the real price per useful answer na three times the value for the chart, and e fit still fail the task. Compare am with your own prompts before you compare price. For agent workloads, the usual answer na to route requests by difficulty and keep the cheap local tokens for bulk work. Na mostly this be wetin to control agent spending on VPS mean.
Billing you forgot about. Hourly GPU instance wey you stop fit continue to charge for attached storage and reserved IP address. Read the invoice, no be only the price page.
When self-hosting dey win for something wey no be price
Four cases wey arithmetic no be the deciding factor.
- Data wey no fit comot from your control. If compliance rule forbid sending the text go third party, price per token no be the question wey dem dey ask.
- Steady high volume on schedule. A batch classification job wey dey run six hours every night dey 25% duty cycle by construction, and e no go ever surprise you with bill.
- Rate limits. Your own card get one queue, and na you own am.
- A model wey no API dey offer. If you need one specific fine-tune, nothing dey wey you fit compare am with.
If you wan test the cheap version first, running small model for VPS with Ollama go cost one afternoon, and sizing open model against the card wey you for rent go tell you which GPU you really need. Measure am there before you sign up for one month of GPU rent.
FAQ
For which monthly token volume GPU VPS go beat API pricing?
Divide the monthly cost of the GPU by the API price for every million output tokens. If one card dey rent for $365 per month, and frontier API charge 15.00 dollars for every million, e go break even at 24.3 million output tokens per month. Against small commercial model wey charge 5.00 dollars, na 73 million. Against hosted open-weight model wey charge 0.20 dollars, one mid-range card no fit generate enough tokens for one month to break even at all.
Why hosted open-weight API dey cost less than my own GPU?
Because dem set the price close to the cost of GPU wey dey fully loaded, but your card no dey fully loaded. Provider wey dey serve thousands of concurrent requests fit keep the fleet close to saturation, so e fit sell tokens near the marginal cost of producing dem. Your card dey idle for most of the day, but you still dey pay for the idle hours. For 10% duty cycle, your cost na 3.47 dollars for every million output tokens, while their own na 0.20 dollars.
I suppose count input tokens together with output tokens?
Count dem if your prompts long. This comparison use output tokens only because na dem dey contribute most for chat and agent work. If you add input, e go change both sides. For the API side, e dey show as separate line for invoice, and the price lower. For your own card, prefill dey use GPU time, so the cost don already dey inside the aggregate tokens per second wey you measure. Measure with your real prompt lengths, and both sides go remain comparable.
How I fit measure the tokens per second wey the formula need?
Serve the model the same way you plan to use am, then read the aggregate generation rate while real concurrency dey run. With Ollama, ollama run <model> --verbose dey print eval rate in tokens per second, but na one stream, so e go understate a batched server. With vLLM, the running server logs Avg generation throughput while requests dey in flight, and na that figure you suppose use. Monitor nvidia-smi at the same time. If GPU utilisation no dey close to 100% during generation, you never find the ceiling yet.
GPU VPS worth renting if utilisation dey below 10%?
No, if na price you dey consider. For 10% duty cycle, you pay 3.47 dollars for every million output tokens, and for 2% you pay 17.36 dollars. Both dey above every metered API for this comparison except the frontier tier. Rent below that line only if privacy or model wey no API dey offer na the thing wey you dey pay for.