Mixed precision: fp16, bf16, fp8

01 · Concept

Concept

Download Qwen3.8-27B and the released safetensors are bf16: about 2 bytes per stored parameter. That fact describes the distributed checkpoint, not the optimizer, master-weight, accumulation, or gradient dtypes used during training. The training recipe is not published. Mixed-precision design must therefore be taught as a set of explicit policies and measured safeguards, not reconstructed as Qwen history from the output dtype.

Start with what a floating-point format is buying. Bits divide among sign, exponent, and fraction. More exponent bits widen dynamic range; more fraction bits sharpen local precision. fp16 spends five bits on exponent and ten explicit bits on fraction: precise near a given magnitude, but with a range so narrow that overflow and underflow are routine in training. bf16 spends eight bits on exponent — the same width as fp32 — and seven explicit fraction bits, so it almost never overflows and is correspondingly coarse.

Coarse how? An fp16 or bf16 value carries an implicit leading one, so bf16’s 7 explicit fraction bits give 8 bits of significand. Near a weight of magnitude 1.0, the smallest representable step is therefore

Now consider a realistic optimizer update. Late in training with a decayed learning rate, a typical parameter change is of order $10^{-5}$ relative to the weight. Add it directly to a bf16 weight of 1.0:

because the next representable bf16 value above 1.0 is 1.0078125, and 1.00001 is nowhere near halfway there. The update does not shrink. It vanishes. Repeat that for 100,000 steps and the weight has still not moved, even though the accumulated intended change was about 1.0 — a full unit of drift, silently discarded one step at a time. This is swamping, and it is the reason mixed-precision training exists.

One conservative fix is a master copy in fp32. Updates accumulate there, where the significand is 24 bits and $10^{-5}$ is representable against 1.0 with room to spare, and the bf16 tensors used in the forward pass are regenerated from it. The storage cost of that master state is 4 bytes per parameter. Lesson 9.3 owns the model-wide byte budget for this course; here the important point is conditional: if a recipe keeps fp32 master weights and two fp32 AdamW moments alongside bf16 parameters and gradients, it reaches about 16 persistent bytes per parameter. Pure-bf16/AnyPrecision variants use different state policies and must establish stability empirically.

Now the comparison that is usually stated wrongly. Training everything in fp32 — 4 bytes each for weights, gradients, and the two moments — also costs 16 bytes per parameter. Under that conservative recipe, mixed precision does not shrink the persistent optimizer footprint. What it buys is the accelerator’s fast reduced-precision matrix units, roughly half the bandwidth on every weight and gradient it moves, and activations stored at 2 bytes rather than 4 — which is the memory that actually dominates a long-context step, as lesson 5.10 works out. Numerically safe alternatives may use different state dtypes, but each needs validation against a wider-precision baseline.

Dynamic loss scaling is the fp16-specific safeguard for the opposite failure. Multiply the loss by a scale before backward so small gradients land inside fp16’s representable range; before the optimizer step, unscale and check for non-finite values, skipping the update and lowering the scale on overflow, raising it cautiously after stable steps. In exact arithmetic the scaling cancels; in finite representation the path matters. bf16’s range usually makes this unnecessary, which is one reason it became the forgiving default on supporting hardware.

fp8 pushes further, with eight total bits in formats trading exponent against fraction — commonly E4M3 and E5M2. Halving weight storage again would take our 54 GB to 27 GB, and supported matrix units go faster still, but raw casting is unsafe. Recipes maintain scales so tensor values occupy the useful representable range, choose formats per operation, accumulate in wider precision, and monitor saturation. Scaling can be per tensor, per channel, or per block: a single tensor-wide scale is cheap but lets one outlier waste most of the range, while finer scales fit varied distributions at the price of metadata and kernel complexity. Delayed scaling reuses recent maxima to pick a scale for later steps, trading responsiveness for stability.

Distribution makes this harder in ways worth anticipating. A gradient can be finite on every rank and overflow during reduction. Communication may use a dtype distinct from optimizer storage. Most subtly, lesson 5.8 partitions one logical tensor across tensor-parallel ranks — and those ranks must agree on its scale, or their partial products are not on a common footing when combined. Checkpoints need enough metadata to resume the whole precision recipe, gradient scaler included.

Some operations stay wide regardless: reductions, softmax normalization, loss computation, RMSNorm statistics, optimizer moments and master weights when the chosen recipe keeps them wide. Numerically stable kernels often accept low-precision inputs and accumulate internally in fp32, so inspect the operation policy rather than inferring precision from the parameter dtype. Validate against a trusted wider-precision baseline, watching loss curves, gradient norms, skipped steps, overflow counts, and downstream metrics — one short run without non-finite values is weak evidence, since rounding differences bend a trajectory gradually rather than breaking it.

The stable rule is to spend bits where they protect information. Narrow formats accelerate large, tolerant matrix work; wider formats accumulate many contributions and hold sensitive state. The bf16 checkpoint you download establishes the released weight dtype only. The precision policy that produced it remains unknown unless the training recipe is published.