Serving in the cloud

01 · Concept

Concept

Everything in this track so far assumed you hold the weights. You picked a quantization rung, sized a KV cache, chose an engine, and accepted that someone on your team owns the process. There is a different arrangement: the model runs on somebody else’s accelerators behind an HTTP endpoint, and you are billed by the token. Qwen3.8-27B is available this way — Cloudflare Workers AI exposes it as @cf/qwen/qwen3.8-27b with vision, reasoning and function calling, priced as of Aug 2026 at USD 0.45 per million input tokens and USD 3.20 per million output tokens. The engineering question is not whether that is cheap. It is what it costs you, for your traffic, compared with the alternative — and that requires actual arithmetic.

Take a concrete workload. A customer-support assistant handles 30,000 conversations a month. Each request carries a 700-token system prompt with product policy plus a 500-token user turn and retrieved context, so 1,200 input tokens, and produces a 400-token reply. Work it through in visible steps.

Input volume: 30,000 requests times 1,200 tokens is 36,000,000 tokens, or 36 million. Output volume: 30,000 times 400 is 12,000,000, or 12 million. Input cost: 36 times USD 0.45 gives USD 16.20. Output cost: 12 times USD 3.20 gives USD 38.40. The gross token-price cost is USD 54.60 before the daily free allocation. Workers AI includes 10,000 neurons free each day, so the invoice also depends on how traffic is distributed across UTC days; a monthly token total alone cannot determine it exactly.

Sit with the shape of that number rather than its size. Output is 12 of the 48 million tokens — a quarter of the volume — and USD 38.40 of USD 54.60, a little over seventy percent of the cost. The price asymmetry is consistent with the prefill-versus-decode asymmetry from lesson 7.3: prefill processes all 1,200 input tokens as one parallel, compute-dense matrix operation, while decode produces 400 tokens strictly one at a time, each step re-reading the resident weights and the growing KV cache. Generation occupies the machine for far longer per token than ingestion does, but Cloudflare does not publish that mechanism as the causal derivation of its price ratio.

The classic wrong turn is to budget with a blended rate. The reasoning: 48 million tokens a month, the price is roughly half a dollar per million, so call it USD 21.60 and move on. That is under the real bill by two and a half times, and it fails in exactly the direction that hurts, because the error grows precisely when a product succeeds at generating more. Never blend. Forecast the two streams separately, because they respond to different product decisions — a longer system prompt moves the cheap number, a chattier assistant moves the expensive one.

That asymmetry has a sharp consequence for this particular model. Qwen3.8-27B ships a thinking mode as its default behaviour, and thinking means the model emits reasoning tokens before its answer. Those are output tokens, billed at output prices. Suppose enabling it adds 800 reasoning tokens per request. Output volume goes from 12 million to 36 million, output cost from USD 38.40 to USD 115.20, and the total bill from USD 54.60 to USD 131.40 — 2.4 times more, from one configuration flag, with the input side untouched. That is not an argument against thinking mode; it is an argument for deciding per route whether a request deserves it, and for knowing that the enable_thinking setting from the model card is a line item on an invoice as well as a quality control.

Now the rent-versus-own comparison, done honestly and without invented hardware prices. Above each day’s free allocation, the managed cost is linear in tokens: double the billed traffic, double the cost, with no capital outlay, no ops rotation, no capacity risk, and no engineer paged when a driver update breaks a kernel. A dedicated deployment — an 80 GB-class accelerator running the vLLM stack from lesson 8.4 — is fixed in cost and capped in throughput: you pay for the machine whether it serves one request or a million, up to the ceiling your KV budget allows, and you pay again in the engineering time that keeps it healthy. If the dedicated option costs C per month all-in, the crossover is C divided by the managed cost of your workload. At a gross USD 54.60 before the daily allowance, essentially no dedicated deployment competes; you would be buying an accelerator to keep it idle. Scale the same traffic pattern a hundredfold and the gross managed cost is USD 5,460 a month, at which point one card and one competent operator plausibly wins — provided your utilisation is high and steady, because a dedicated machine’s economics live entirely on utilisation.

Three factors move that crossover and none of them appear on a price list. Utilisation: spiky traffic wastes owned capacity and costs nothing extra on a meter. Ops capacity: a team of three shipping features has a different real cost for running inference than a platform team of fifteen. And token efficiency, which is where lesson 1.4 earns its place as a prerequisite — the same assistant serving Portuguese users bills more per conversation than one serving English users, because byte-level BPE fragments under-represented text into more tokens for identical meaning. On a metered endpoint, tokenizer behaviour is a line item.

Lesson 8.12 puts this option beside every local and self-hosted stack in the track and gives you one framework for choosing among them.