GPU VPS vs API tokens: the break-even point

Where the break-even actually sits

A GPU VPS beats per-token API billing at one point: when the fixed monthly rent, divided by the output tokens you actually generate that month, falls below what the API charges for the same tokens. The rent does not move. The API bill moves with every request. So the answer is always a monthly volume, never a plain yes or no.

Work it on a mid-range GPU VPS at $0.50 per hour, which is $365 for a 730-hour month. Against a frontier model API you break even at 24.3 million output tokens a month. Against a small commercial model it is 73 million. Against a hosted open-weight model of the same size you never break even, because one card cannot produce enough tokens in a month to reach the crossing point.

Published break-even studies do not answer this question. Two of them from earlier in 2026 put the crossover near 72% sustained utilisation on an H200, and between 22% and 48% duty cycle on an MI300X. Both compare against the same vendor's own serverless product, and both price accelerators that cost more per hour than most readers here spend in a month. The arithmetic is identical. What follows redoes it for one card, one open model in the 7B to 30B range, and ordinary metered API billing.

Every number below is an input, not a result. Replace all of them with your own.

The formula, so you can substitute your own numbers

cost_per_million = (hourly_rate * 1000000) / (tokens_per_second * 3600 * duty_cycle)

breakeven_millions_per_month = (hourly_rate * hours_per_month) / api_price_per_million

capacity_millions_per_month = (tokens_per_second * 3600 * hours_per_month) / 1000000

required_duty_cycle = breakeven_millions_per_month / capacity_millions_per_month

Four inputs, and you can measure or look up all four.

• hourly_rate is what the GPU VPS costs per hour, counting the hours it sits idle. If you pay monthly, divide the monthly price by 730.
• tokens_per_second is the aggregate output rate your server sustains under your real concurrency. It is not the single-stream number from a vendor chart.
• duty_cycle is the fraction of the month the GPU spends generating tokens. A box you rent all month and use for two hours a day sits at 8.3%.
• api_price_per_million is the metered price you are comparing against, for output tokens.

The example prices output tokens on both sides, because output dominates the bill for chat and agent work. If your prompts are long, add input to both sides. On the API side that is a separate line on the invoice. On your own card, prefill consumes GPU time, so it already shows up as a lower measured tokens_per_second.

How to measure tokens per second before you trust the arithmetic

Everything above rests on one measured number. Get it wrong by a factor of three and the answer is wrong by a factor of three. Measure it on the card you are renting, with the model and the quantisation you will really run.

Ollama gives you the single-stream figure in one command:

ollama run qwen3:8b --verbose "Write 400 words about disk latency."

--verbose prints a timing block after the answer. The line that matters is eval rate, in tokens per second, which counts generation only. prompt eval rate is prefill speed and is normally much higher. Your numbers will differ from these:

eval count:       412 token(s)
eval duration:    9.612s
eval rate:        42.86 tokens/s

Single stream is the wrong number for a cost model, because it measures one request at a time on a card that can serve many at once. For the aggregate figure, serve the model with vLLM and read the throughput the server reports about itself:

pip install vllm
vllm serve Qwen/Qwen3-8B --max-model-len 8192

While requests are in flight, the server logs a status line on every reporting interval. The exact fields move between vLLM releases, so read yours rather than mine:

Avg prompt throughput: 812.4 tokens/s, Avg generation throughput: 396.1 tokens/s, Running: 16 reqs, Waiting: 0 reqs, GPU KV cache usage: 21.7%

Avg generation throughput is the number the formula wants. It climbs as you add concurrent requests until the KV cache (key-value cache, the per-request attention state vLLM keeps in VRAM) is full, then it stops climbing. Push past that and requests queue instead of going faster, which you see as a rising Waiting count. vLLM also ships a load generator, vllm bench serve. Its flags change between versions, so run vllm bench serve --help on the version you installed instead of copying a command from a blog post.

Watch the card while the test runs:

nvidia-smi --query-gpu=utilization.gpu,memory.used,power.draw --format=csv -l 5

If utilization.gpu sits near 100% during generation, you are throughput-bound and the number you measured is the real ceiling. If it stays low, something else is the limit: too few concurrent requests, a slow client, or a model that does not fit in VRAM and is being partly offloaded to system RAM. Ollama and vLLM make very different trade-offs here, and the gap between them on the same card is wide enough to move your break-even by a factor of several.

What the bill looks like across a month

The worked example is one 24 GB GPU VPS at $0.50 per hour, an 8B open model served by vLLM, measured at 400 output tokens per second aggregate with 16 concurrent requests. Rent is fixed whether you use the card or not. Capacity at that rate is 1,051 million output tokens per month, which is what the card produces if it never stops.

[
  {
    "output_tokens_millions": 5,
    "gpu_vps_usd": 365,
    "open_api_usd": 1,
    "small_api_usd": 25,
    "frontier_api_usd": 75
  },
  {
    "output_tokens_millions": 10,
    "gpu_vps_usd": 365,
    "open_api_usd": 2,
    "small_api_usd": 50,
    "frontier_api_usd": 150
  },
  {
    "output_tokens_millions": 25,
    "gpu_vps_usd": 365,
    "open_api_usd": 5,
    "small_api_usd": 125,
    "frontier_api_usd": 375
  },
  {
    "output_tokens_millions": 50,
    "gpu_vps_usd": 365,
    "open_api_usd": 10,
    "small_api_usd": 250,
    "frontier_api_usd": 750
  },
  {
    "output_tokens_millions": 100,
    "gpu_vps_usd": 365,
    "open_api_usd": 20,
    "small_api_usd": 500,
    "frontier_api_usd": 1500
  },
  {
    "output_tokens_millions": 250,
    "gpu_vps_usd": 365,
    "open_api_usd": 50,
    "small_api_usd": 1250,
    "frontier_api_usd": 3750
  },
  {
    "output_tokens_millions": 500,
    "gpu_vps_usd": 365,
    "open_api_usd": 100,
    "small_api_usd": 2500,
    "frontier_api_usd": 7500
  },
  {
    "output_tokens_millions": 1000,
    "gpu_vps_usd": 365,
    "open_api_usd": 200,
    "small_api_usd": 5000,
    "frontier_api_usd": 15000
  }
]

The GPU line is flat at 365 dollars because rent does not care what you do with the card. Every API line is a straight line through zero. Each pair crosses exactly once.

At 25 million output tokens a month the frontier API bills 375 dollars, so the two sides are within ten dollars of each other. At 50 million the small commercial model bills 250 dollars and is still the cheaper option. At 1000 million output tokens, which needs the card busy 95% of the month, the hosted open-weight API bills 200 dollars against the same rent. The card loses by a factor of nearly two at the exact volume where it is working hardest.

That last result surprises people, and it is not an accident. A hosted open-weight endpoint is a GPU fleet run at high utilisation, so its price sits close to the cost of a saturated card. You cannot beat a saturated fleet by renting one card and running it at less than full load. What you can beat is frontier pricing, which is set by capability rather than by silicon time.

Cost per million output tokens at each duty cycle

Volume and duty cycle are the same fact seen from two sides. Rent buys hours. Idle hours produce nothing and still cost money.

[
  {
    "label": "100% duty",
    "self_host_usd_per_million": "0.35",
    "open_api_usd_per_million": "0.20",
    "small_api_usd_per_million": "5.00",
    "frontier_api_usd_per_million": "15.00"
  },
  {
    "label": "50% duty",
    "self_host_usd_per_million": "0.69",
    "open_api_usd_per_million": "0.20",
    "small_api_usd_per_million": "5.00",
    "frontier_api_usd_per_million": "15.00"
  },
  {
    "label": "25% duty",
    "self_host_usd_per_million": "1.39",
    "open_api_usd_per_million": "0.20",
    "small_api_usd_per_million": "5.00",
    "frontier_api_usd_per_million": "15.00"
  },
  {
    "label": "10% duty",
    "self_host_usd_per_million": "3.47",
    "open_api_usd_per_million": "0.20",
    "small_api_usd_per_million": "5.00",
    "frontier_api_usd_per_million": "15.00"
  },
  {
    "label": "5% duty",
    "self_host_usd_per_million": "6.94",
    "open_api_usd_per_million": "0.20",
    "small_api_usd_per_million": "5.00",
    "frontier_api_usd_per_million": "15.00"
  },
  {
    "label": "2% duty",
    "self_host_usd_per_million": "17.36",
    "open_api_usd_per_million": "0.20",
    "small_api_usd_per_million": "5.00",
    "frontier_api_usd_per_million": "15.00"
  }
]

The three API columns are typical published list prices as of August 2026: 0.20 dollars per million output tokens for a hosted 8B open-weight model, 5.00 dollars for a small commercial model, and 15.00 dollars for a frontier model. They are illustrative. Check today's price page before you decide anything, and if your comparison is against a flat monthly plan rather than metered tokens, the subscription arithmetic works differently and the crossing point moves again.

Run the card flat out and a million output tokens cost 0.35 dollars, which is genuinely cheap. At 10% duty cycle the same million costs 3.47 dollars. At 2% duty cycle it costs 17.36 dollars, which is not in the same range as the 0.20 dollars a hosted open model charges for identical output.

Below roughly 10% duty cycle, renting a GPU is the expensive choice. You are paying 3.47 dollars per million tokens for output that sells for 0.20 dollars, and what you are buying with the difference is privacy and a bill that does not move. Those can be worth real money. They are not a price win, so do not file them as one.

The break-even volume for each API tier

[
  {
    "label": "Hosted open 8B API",
    "breakeven_tokens_millions": 1825,
    "required_duty_pct": 174
  },
  {
    "label": "Small commercial model",
    "breakeven_tokens_millions": 73,
    "required_duty_pct": 6.9
  },
  {
    "label": "Frontier model",
    "breakeven_tokens_millions": 24.3,
    "required_duty_pct": 2.3
  }
]

Against the frontier tier you need 24.3 million output tokens a month, which is only 2.3% of what the card can do. That is a low bar. A small team running coding agents through the working day clears it.

Against the small commercial tier you need 73 million tokens a month, or 6.9% duty cycle. Against the hosted open-weight tier the required duty cycle is 174%. Anything above 100% is unreachable by definition: the card would have to run more hours than the month contains. One mid-range card at this hourly rate cannot win that comparison, so the only ways to change the result are a cheaper card, a faster card, or a reason that is not price.

What the formula hides

The formula prices GPU hours and tokens. Several real costs sit outside it.

Cold starts. An 8B model in 16-bit weights is about 16 GB, and loading it from local disk into VRAM takes tens of seconds. Stop the box between uses to save rent and you pay that wait on the first request every time. Leave it running to avoid the wait and your duty cycle collapses, which pushes the cost per token up. That trade is the entire reason serverless inference exists.

Storage and download. Weights are large. An 8B model at 16-bit is around 16 GB, a 30B model quantised to 4-bit is around 18 GB, and a 30B model at 16-bit does not fit on a 24 GB card at all. The ceiling gets stark quickly at the top end, where running a trillion-parameter open model like Kimi K3 means the weights alone outgrow any single card you can rent by the hour. You pay for that disk every month, and you pay in time on every rebuild. Run du -sh ~/.cache/huggingface/hub after a week of experiments. It grows faster than you expect, because every quantisation you tried once is still sitting there.

Your own time. Driver and CUDA versions, out-of-memory errors at a context length that worked yesterday, a model update that changes the chat template. None of it appears in a cost-per-token figure, and all of it is billed to your evenings. If you have not sized one of these boxes before, what a GPU VPS actually gives you is worth reading before you commit to a month of rent.

The quality gap. This is the largest hidden cost and the hardest to price. An 8B open model is not a frontier model. If it needs three attempts where the frontier model needs one, its real price per useful answer is three times the chart value, and it may fail the task anyway. Compare on your own prompts before you compare on price. For agent workloads the usual answer is to route by difficulty and keep the cheap local tokens for bulk work, which is most of what controlling agent spend on a VPS comes down to.

Billing you forgot about. An hourly GPU instance you stopped often keeps billing for its attached storage and its reserved IP address. Read the invoice, not the price page.

When self-hosting wins on something other than price

Four cases where the arithmetic is not the deciding factor.

• Data that cannot leave your control. If a compliance rule forbids sending the text to a third party, price per token is not the question being asked.
• Steady high volume on a schedule. A batch classification job that runs six hours every night sits at 25% duty cycle by construction, and it never surprises you with a bill.
• Rate limits. Your own card has one queue and it belongs to you.
• A model no API offers. If you need a specific fine-tune, there is nothing to compare against.

If you want to test the cheap version first, running a small model on a VPS with Ollama costs an afternoon, and sizing an open model against the card you would rent tells you which GPU you actually need. Measure there before you sign up for a month of GPU rent.

FAQ

At what monthly token volume does a GPU VPS beat API pricing?

Divide the monthly cost of the GPU by the API price per million output tokens. A card renting for $365 a month, against a frontier API at 15.00 dollars per million, breaks even at 24.3 million output tokens per month. Against a small commercial model at 5.00 dollars it is 73 million. Against a hosted open-weight model at 0.20 dollars, one mid-range card cannot generate enough tokens in a month to break even at all.

Why does a hosted open-weight API cost less than my own GPU?

Because its price is set close to the cost of a fully loaded GPU, and your card is not fully loaded. A provider serving thousands of concurrent requests keeps its fleet near saturation, so it can sell tokens near the marginal cost of producing them. Your card is idle for most of the day and you pay for the idle hours. At 10% duty cycle your cost is 3.47 dollars per million output tokens against their 0.20 dollars.

Should I count input tokens as well as output tokens?

Count them if your prompts are long. The comparison here uses output tokens only, which is the dominant term for chat and agent work. Adding input changes both sides. On the API side it is a separate, lower-priced line on the invoice. On your own card, prefill consumes GPU time, so the cost is already inside the aggregate tokens per second you measured. Measure with your real prompt lengths and the two sides stay comparable.

How do I measure the tokens per second the formula needs?

Serve the model the way you will use it, then read the aggregate generation rate under real concurrency. With Ollama, ollama run <model> --verbose prints an eval rate in tokens per second, but that is a single stream and understates a batched server. With vLLM, the running server logs Avg generation throughput while requests are in flight, and that is the figure to use. Watch nvidia-smi at the same time. If GPU utilisation is not close to 100% during generation, you have not found the ceiling yet.

Is a GPU VPS worth renting below 10% utilisation?

Not on price. At 10% duty cycle you pay 3.47 dollars per million output tokens, and at 2% you pay 17.36 dollars. Both sit above every metered API in this comparison except the frontier tier. Rent below that line only when privacy or a model no API offers is what you are paying for.