Cerebras and wafer scale

01 · Concept

Concept

Lesson 9.1 ended on an uncomfortable number. For a measured batch-one path, the bandwidth roofline is

with $B$ the memory bandwidth and $W$ the actual bytes read per token. Using the full 54 GB checkpoint as a stated proxy, and an H100 SXM’s reported 3.35 TB/s of HBM3 gives roughly 60 tokens per second as a simple checkpoint-size estimate. Every subsequent lesson in this track has accepted that structure and argued about the numerator or the denominator. This lesson is about a design that attacks the numerator B by changing the memory technology.

The reason HBM bandwidth is where it is comes down to where the memory sits. High-bandwidth memory is DRAM stacked beside the processor die and reached through an interposer — vastly better than a socketed DIMM, but still an off-die interface with a finite number of wires running at a finite rate. SRAM, the memory used for on-chip caches, sits in the same silicon as the arithmetic and is reached without leaving the die. Its bandwidth is orders of magnitude higher, and its capacity per unit of silicon area is orders of magnitude lower. That single trade is the whole story of every cache hierarchy ever built: a small fast tier in front of a large slow one, with a hit rate deciding which one you actually feel.

Wafer-scale processing takes the trade to its logical extreme. Instead of cutting a wafer into hundreds of separate chips, the entire wafer is fabricated as one device, tiled with a very large number of small cores each carrying its own local SRAM, connected by an on-wafer fabric. Cerebras describes this architecture and the inference service built on it in its product and documentation pages. The consequence for language-model decode is direct: if the model’s weights are distributed across that on-wafer SRAM, the per-token weight read never touches an HBM interface, and the numerator B in lesson 9.1’s roofline is replaced by something much larger.

How much larger? Rather than quoting a bandwidth figure and asking you to trust it, invert the formula and turn a public throughput claim into a bandwidth requirement. Vendors in this space advertise rates above a thousand tokens per second on large models. Take a stated scenario in which 54 GB are active per token:

Fifty-four terabytes per second — about sixteen times an H100’s HBM3 figure. Using the pinned 17.1 GB Q4_K_M file size only as a proxy the requirement softens but stays severe:

Still about five times HBM3, but only as an artifact-size sanity check; exact active bytes require profiling. This is the useful discipline: a throughput claim implies a bandwidth, and you can check whether the implied bandwidth is plausible for the memory technology in question. From stacked DRAM these rates are not merely difficult, they are outside the interface’s physics. From on-wafer SRAM they are dimensionally consistent. That does not make them true — it makes them the kind of claim worth investigating rather than dismissing.

Now the wrong turn, which is the reason this lesson sits before the cost capstone rather than after it. A reader computes that a wafer-scale system is roughly sixteen times faster per stream than a GPU and concludes it must therefore be roughly sixteen times cheaper per token. Two errors are stacked there. The first is treating a roofline bound as a throughput prediction: B/W is an upper bound only when W is actual active traffic, and achieved rates depend on kernel efficiency, compiler quality, the fabric’s behaviour under the model’s actual dataflow, and whether every operator — including this model’s Gated DeltaNet layers — maps onto the hardware at all. The second error is comparing the wrong quantity. Nobody serves a production GPU at batch one. Lesson 7.3 showed that batching amortizes the weight read across many concurrent sequences, so a GPU’s aggregate throughput climbs far above its single-stream ceiling while each user still sees roughly single-stream latency. The GPU’s product is throughput per dollar; the wafer’s product is latency per stream. Those are different goods, and lesson 9.10 will show that cost per token is set by system cost per second divided by tokens per second, never by a speed ratio alone.

The durable takeaway is a method, not a verdict on a vendor. The bandwidth wall of lesson 9.1 is not a law about GPUs, it is a law about the ratio of supply rate to bytes moved — and any architecture that changes where the weights physically sit changes that ratio. Wafer scale is one such architecture; quantization is another, reducing the denominator W; the hybrid Gated DeltaNet layout of lesson 4.16 reduces growing KV-history traffic at long context rather than the model-weight term. Cost decides which one you can afford, and that is the last lesson of the course.