---
title: "Fifty Cents on the Dollar: Why the Token Bill Is Coming Due"
url: "https://bitrefinery.com/blog/fifty-cents-on-the-dollar-token-bill-coming-due"
description: "Frontier tokens are running 20–100x the price of open models, and CFOs are starting to notice. Here's the cost math on running your own GPU and harness, from someone who does it for a living."
author: "Mike George, VP of Operations"
date: "2026-07-18"
lastmod: "2026-07-27"
tags: ["gpu-hosting", "private-llm", "cost-analysis", "infrastructure-economics", "token-economics"]
source: "blog CMS"
---

# Fifty Cents on the Dollar: Why the Token Bill Is Coming Due

I run infrastructure for a living, so I spend a lot of time watching the gap between what people *pay* for compute and what compute actually *costs*. Right now that gap is the widest I've seen it in seventeen years, and it's all hiding inside one line item: tokens.

There was a segment on the All-In podcast recently that put numbers to something we've been telling customers for months. The hosts were reading off per-million-token prices across the market, and the spread is almost comical. A million input tokens on a frontier model runs somewhere in the $26–$56 range. The same work on a good open model runs closer to **fifty cents**. Elon's models were quoted around a dollar. The Chinese open models, fifty cents. One of the hosts said the quiet part out loud: people paying $26 to $56 "should be paying 50 cents."

He's basically right. And the reason that's not just a hot take is that I can show you the math on the fifty-cent side, because it's the business I'm in.

## The bill nobody's watching

The setup for all of this is a problem that's been building quietly. Ramp — the corporate card company — launched a token-spend management product this month, and the CEO mentioned on air that token spend among their customers has grown **21x** over the last year. Not 21 percent. Twenty-one times.

That number should scare any CFO, because token spend has three properties that make it uniquely dangerous as a budget line:

1. **It's a tab, not a plan.** Your engineers can spend as much as they want, and the meter runs in the background. Most finance teams can't see it proactively.
2. **The rates drift up, not down.** Every time a lab ships a new flagship, the default model your team reaches for gets more expensive per token, and the newest tokenizer often generates *more* tokens for the same text.
3. **The person spending isn't the person paying.** The engineer wants the best model for the task. They're not tied to the P&L. So they reach for the frontier model on work a much cheaper model would finish just fine.

That last one is the whole game. On the podcast, one host guessed that **98%** of his frontier-model prompts were things a small open model could have handled. That's not a rigorous number, but it rhymes with what we see. The overwhelming majority of production AI work — classification, extraction, routing, summarization, retrieval-augmented answering, first-draft generation, internal tooling — does not need a frontier model. It needs a *competent* model that's up all the time and costs almost nothing to run.

The frontier labs know this. It's why the same host pointed out that the economics only work in their favor if you keep sending them the easy stuff at premium rates.

## Where the fifty cents actually comes from

Here's the part I can speak to directly, because it's our day job.

When you pay a hosted API fifty cents a million tokens for an open model, you're renting a slice of somebody's GPU. When you pay $26–$56, you're renting a slice of a frontier model *plus* a very large margin *plus*, in a lot of cases, the privilege of sending your proprietary data through someone else's stack. The fifty-cent number is the honest floor — it's roughly what the compute costs plus a thin margin. So the interesting question isn't "why is fifty cents possible." It's "what happens when you own the GPU instead of renting the slice?"

Let me walk through it with our own numbers, because I'd rather you check my work than take my word.

We host single-tenant NVIDIA RTX 6000 Ada cards (48 GB) at a flat **$790 a month**. That's the whole cost. No egress fees, no per-token meter, no usage surprise at the end of the quarter. One card, yours, running whatever open model you want behind your own firewall.

Now the throughput. Published inference benchmarks for a 48 GB Ada-class card running a batched vLLM server land in these bands, and I'm deliberately using the conservative end (the Ampere-generation A6000 numbers, since the Ada card is faster and its published inference numbers are sparse — I'd rather understate it):

- A mid-size model (14–32B) under modest batching: **~600 tokens/sec sustained**
- A smaller model (8–14B) under healthy batching: **~1,200 tokens/sec**
- A small or MoE model (7–8B) under high concurrency: **~2,400 tokens/sec**

Turn those into monthly token volumes and divide by the $790:

| Workload | Sustained throughput | Tokens/month (100% util) | Blended cost per million |
|---|---|---|---|
| Mid-size (14–32B) | ~600 tok/s | ~1,577 M | **$0.50** |
| Small (8–14B) | ~1,200 tok/s | ~3,154 M | **$0.25** |
| Small / MoE (7–8B) | ~2,400 tok/s | ~6,307 M | **$0.13** |

![Blended cost per million tokens on a log scale, comparing frontier API pricing against a flat-rate single-tenant RTX 6000 Ada at several model sizes and duty cycles](/api/storage/files/blog-images/blog-1784383588120.png)

Look at the top row. A mid-size open model on one owned card lands you at **fifty cents a million tokens** — the exact number the podcast said you *should* be paying. Not close to it. It. Move to a smaller model your workload probably doesn't need to outgrow, and you're at a quarter, or thirteen cents.

Compare that to $26–$56 on the frontier. That's not a 2x savings. That's **50x to 100x**.

## "But nobody runs a GPU at 100%"

Correct, and I'm not going to pretend otherwise, because the whole point of doing this honestly is that it holds up when you run the numbers yourself.

Real workloads are spiky. Nights and weekends go quiet. So let's redo the table at a **30% average duty cycle** — the card sitting idle 70% of the time:

- Mid-size (14–32B) at 30% utilization: **~$1.67 per million tokens**
- Small (8–14B) at 30% utilization: **~$0.84 per million tokens**
- Small / MoE at 30% utilization: **~$0.42 per million tokens**

Even with the card idle most of the time, running a mid-size model yourself is **$1.67 against $26–$56**. Push it further: at a *5%* duty cycle — the machine effectively off — you're still at about $10 per million. You'd have to leave it switched off 95% of the month before the economics stopped being lopsided in your favor.

And here's the break-even that actually matters. Against a conservative $10-per-million blended API rate — well below the frontier headline, so I'm arguing against myself here — you only need to push about **80 million tokens a month** to justify the $790 box. Measure it against real frontier pricing and the bar drops to roughly 30 million. If you're doing any real agentic work — where a single task can push 400K to 2M tokens through the model — you blow past either number in days, not months. The card pays for itself almost immediately, and everything after that is close to free.

That's the "own the base" half of how we think about infrastructure. You own a fixed, predictable slab of compute for the workloads you run constantly, and you rent the spike — the genuine frontier reasoning task, the occasional hard problem — from whoever's best at it that month.

## The harness is the other half

Cheap tokens are only half the story. The other half is what the industry's started calling the **harness** — the layer that sits between your application and the model, deciding what runs where.

This is the more important structural shift, and it's why "run your own" is the future rather than just a cost hack. A few things happened at once:

- Open models got genuinely good. The gap between the frontier and the best open weights is now measured in months, sometimes weeks — and for the 98% of tasks that aren't at the frontier, there's no meaningful gap at all.
- The serving stacks (vLLM, SGLang) matured to where batched throughput on a single card is production-grade.
- The coding harnesses started going open-source. The harness is turning into a commodity you control rather than a black box you rent, and once you control it, where a request runs becomes your decision instead of a vendor's default.

Put those together and you get something that didn't exist eighteen months ago: a **model-routing layer you own**, sitting on GPUs you own, sending the easy 98% to a cheap open model on your own hardware and only escalating the genuinely hard requests to a frontier API. Own the base, route the spike. Every token you keep in-house is a token that costs you thirteen cents instead of fifty dollars — and never leaves your building.

Which brings me to the part I care about most.

## You're not just paying with money

The most expensive thing about the frontier-API default isn't the fifty dollars. It's what you hand over to earn the privilege of paying it.

When your team runs everything through a hosted frontier model, you're feeding that model your proprietary context — your documents, your code, your customer data, your internal reasoning. Even with a zero-data-retention policy, the honest engineering answer is that privacy in these pipelines is brittle. A retention setting is a policy promise about a system you can't inspect, enforced by a company whose incentives aren't yours. It's not a technical boundary. Those two things get conflated constantly, and the difference only becomes visible on the day it matters.

So there's a compounding cost. You pay the premium in dollars, *and* you pay it in exposure — feeding your alpha into a stack you don't control. It's a double whammy: more expensive, and you're mortgaging your data sovereignty to get it.

The answer to that isn't a better promise from a bigger vendor. It's a real trust boundary. Single-tenant hardware. Your model, your weights, your data, inside your environment, with nothing phoning home. That's not a feature we bolt on — it's the entire architecture. When you own the card, there's no telemetry vector to worry about, because there's no third party in the path to leak to.

## What this looks like at Bit Refinery

I'll keep this part short, because if the math above didn't land, a sales pitch won't fix it.

We host single-tenant NVIDIA GPUs in Colorado — RTX 6000 Ada 48 GB at $790/mo and RTX PRO 5000 Blackwell 48 GB at $840/mo, up to 7 cards per pod with NVLink — at flat monthly pricing with zero egress fees and 10 TB included. You get root access, full isolation, no noisy neighbors, and no shared scheduler. You run open models (Qwen, Llama, Mistral, DeepSeek, Gemma, and others) behind your own firewall, on hardware that's yours for the month. Your data never leaves that environment, and it's never trained on.

For the workloads you run constantly — the internal assistant, the RAG system, the document pipeline, the coding harness — that's the base you own. For the occasional frontier task, keep your API key and route the spike. The point isn't to never touch a frontier model. The point is to stop paying frontier rates for work that a fifty-cent token finishes just as well.

If you want to size a pod against your own numbers, [the Private GPU Cloud page](/services/private-gpu-cloud) has the current configurations and a calculator that'll do the arithmetic for you.

The token bill is coming due. Ramp built a product because CFOs asked them to, and at 21x annual growth, somebody's going to miss a quarter explaining a compute line nobody was watching. When that conversation reaches your finance team, the question won't be "which model is best." It'll be "why are we paying fifty dollars for work that costs fifty cents."

Have a good answer ready. Better yet, own the answer.

---

*Mike George is VP of Operations at Bit Refinery, a Denver-based private GPU and hybrid-cloud infrastructure company. We build and operate single-tenant GPU compute in Colorado — flat monthly pricing, zero egress, your data stays yours. If you want to pressure-test the numbers above against your own workload, that's a conversation we're always happy to have.*

*Cost figures are built from published API pricing and community inference benchmarks as of July 2026, using the conservative end of throughput ranges. Your mileage will vary with model choice, batching, and utilization — run your own numbers. That's the whole idea.*