NVIDIA, CUDA, Hopper and Blackwell

01 · Concept

Concept

A procurement question arrives with a deadline: the company will serve Qwen3.8-27B, and the choice is between renting 80 GB Hopper-generation accelerators and paying substantially more for 192 GB-class Blackwell-generation ones. The vendor material compares them on arithmetic throughput, which lesson 9.1 established is the currency single-stream decoding does not spend. The decision turns instead on two lines from lesson 9.3, capacity and bandwidth, and this lesson works both out.

First, the shape of the machine, because the vocabulary recurs in every profiling report. A CUDA-programmable GPU is a grid of streaming multiprocessors, each a scheduler over many arithmetic lanes plus a slab of software-managed shared memory and a register file. An H100 SXM part carries on the order of 130 of them, a vendor figure as of Aug 2026 that varies by SKU and binning. Inside each SM sit tensor cores, fixed-function units that consume small matrix tiles and emit accumulated products, and essentially all of a transformer’s arithmetic lands there. Around the SMs sits the memory hierarchy that the roofline of lesson 9.2 is drawn against: registers, shared memory and L1, a shared L2, and then high-bandwidth memory stacked beside the die. Hopper added a fourth generation of tensor cores with an FP8 path and per-tensor scaling logic, and Blackwell added a fifth generation extending that to FP4. The engine builders of lesson 8.6 exist largely to select which of these paths each layer takes.

Now the arithmetic that decides the purchase. Take the Hopper part first, reusing the budget from lesson 9.3 without re-deriving it. An 80 GB part reports roughly 79.6 GiB usable; the bf16 weights of Qwen3.8-27B are about 54 GB, which is 50.3 GiB; a runtime reserve near 6 GiB covers context, activations, and fragmentation. The pool is about 23.3 GiB, which at 656 MiB per 8,192-token sequence, cache plus the constant DeltaNet state, supports about 36 concurrent conversations, and holds exactly one sequence at the full 262,144-token native context. That is a tight fit, and it is why deployments on this part reach for FP8 weights or split the model across two cards, which drags in the interconnect costs of lesson 9.9.

Now the Blackwell-class part, at a vendor-reported 192 GB of HBM3e. Usable capacity is about 178.8 GiB. The weights cost the same 50.3 GiB, because weights are a fixed charge that does not scale with the card, and a slightly larger reserve of 8 GiB leaves

for the pool. At 656 MiB per 8,192-token sequence that is about 188 concurrent conversations, and at 16,528 MiB per full-native-context sequence it is about seven. Concurrency rose roughly fivefold from a capacity increase of about 2.4 times, and the leverage is entirely because the fixed weight charge is paid once on either card. Every additional gibibyte on the larger part goes to users. This non-linearity is the single most important fact in accelerator selection for a model of this size, and it is invisible in any comparison built on arithmetic throughput.

Bandwidth is the second axis and answers a different question: not how many users, but how fast each one reads. Applying the lesson 9.1 bound to a vendor-reported figure near 8 TB/s,

against about 62 on the Hopper part. For an unchanged active-byte workload, roughly 2.4 times the bandwidth raises the bandwidth roof by roughly 2.4 times. The 54 GB quotients above are checkpoint-size heuristics from vendor claims, not measured throughput or hard limits.

Here is the classic wrong turn, and it usually appears in the same meeting. Someone notes that the newer part supports FP4 and concludes that inference will therefore be twice as fast again, for free. Separate the two effects. A validated FP4 representation may reduce active weight traffic far below bf16, but its exact active-byte reduction depends on the representation and runtime. That can improve bandwidth-limited decode, but the pinned 17.1 GB Q4_K_M GGUF is not evidence for a server FP4 layout. FP4 tensor cores also raise the flat compute roof, which accelerates prefill and time to first token. That is an arithmetic effect and it is also real, but it helps a different phase. Neither is free: the format must have a validated quantization recipe for this model, and Qwen3.8-27B is a hybrid architecture whose 48 Gated DeltaNet layers are newer than most kernel libraries, so FP4 coverage for them should be verified in the specific stack version rather than assumed from a marketing table. Quality regression testing against the model card’s own benchmark set is the minimum before a format change ships.

The habit worth keeping is to price a part on the two lines that decide your workload: how much pool remains after a fixed weight charge, and how many bytes per second cross the threshold. Lesson 9.5 takes the same two lines to a vendor whose entire pitch is the first one.