Cost per token

01 · Concept

Concept

You have a product. It sends Qwen3.8-27B about 200 million input tokens and receives about 40 million output tokens per month — a support assistant with real usage, five parts prompt and context to one part answer. Someone has to decide how it is served, and every argument in that meeting will be a proxy for a number nobody has computed. This lesson computes it three ways, and the discipline it teaches matters more than the winner: state every assumption inline, so that when a price or a throughput figure changes, the reader can redo the whole calculation without you.

Option one, the managed per-token endpoint. Cloudflare Workers AI lists the model as a first-party offering, and as of August 2026 its published prices are USD 0.45 per million input tokens and USD 3.20 per million output tokens.

Note the shape as much as the size: the bill is proportional to traffic, so a quiet month costs nearly nothing and a viral week costs proportionally more. There is no capacity to plan, no upgrade path to negotiate, and no operational surface at all — which is the argument of lesson 8.12 restated as an invoice.

Option two, a rented data-center GPU running vLLM. Two assumptions, both stated rather than quoted: an 80 GB-class instance at USD 2.50 per hour, and a sustained aggregate output rate of 800 tokens per second under continuous batching. Neither is a vendor quote; substitute your own from a real provider page and a real benchmark.

Here is the classic wrong turn, and it is the reason this lesson exists. The engineer divides rental by capacity, gets

compares it against USD 3.20, announces a 3.7-fold saving, and proposes owning the serving stack. The arithmetic is correct and the conclusion is wrong, because you do not rent tokens, you rent seconds. The workload needs 40 million output tokens, so the instance runs at

roughly fourteen times the endpoint price. The saving was real but it belonged to a workload eight times larger than this one. Find the crossover explicitly: one workload unit of 5 million input plus 1 million output costs the endpoint

so the rented GPU wins above roughly 1.7 billion input and 335 million output tokens per month — which, reassuringly, is only about 16 percent of the assumed capacity, so the configuration is feasible rather than self-contradictory. Lesson 9.3’s memory budget is what makes that capacity claim checkable at all: 54 GB of weights against an 80 GB card leaves room for a bounded number of concurrent sequences at 64 KiB per cached token, and concurrency is precisely what turns a memory budget into the throughput number in the denominator.

Option three, owned Apple hardware. Assume a large unified-memory Mac at USD 6,000, amortized straight-line over 36 months, drawing 150 watts on average across the powered-on month, at USD 0.25 per kilowatt-hour.

Lesson 9.6 gives only an artifact-size heuristic near 47 tokens per second, which cannot establish capacity. For this scenario assume a measured sustained 25 output tokens per second: about 65.7 million output tokens per month, making the 40-million-token workload feasible at roughly 61 percent duty, provided requests do not arrive in bursts that a single-stream machine cannot absorb.

Line the three up and the result is genuinely instructive: USD 218 gross managed before the daily allowance, USD 1,825 rented, USD 194 owned. The managed endpoint and the owned Mac are within twelve percent of each other, a rounding error against the difference in everything else, while the rented GPU loses by an order of magnitude purely on utilization.

The general form is worth memorizing because it survives every price change: per-token cost is system cost per unit time divided by tokens produced per unit time, and every lesson in this track fed one of those two terms. Bandwidth and the roofline set the achievable rate; the KV-cache budget sets concurrency, which sets aggregate rate; quantization moves both; the interconnect decides whether extra chips actually add throughput; and utilization — the term nobody puts on a slide — decides whether fixed costs are a bargain or a liability.

That is the end of the course. You began with a function from context to a distribution over 248,320 tokens and you can now trace that function end to end: how text becomes vectors, how attention and Gated DeltaNet mix them, why 16 of 64 layers cache 64 KiB per token while the other 48 hold a flat 144 MiB in the reference float32 path, how the weights were trained and aligned, how a serving stack schedules them, which silicon reads them fastest, and what a month of that costs. More usefully, you can now size a deployment you have never seen — read a config file, derive its memory and bandwidth demands, choose a stack, pick hardware, and defend the choice with arithmetic instead of vendor slides. That skill outlives this model, and the next one will be waiting for it.