Probability you actually need

01 · Concept

Concept

Here is a concrete situation before any vocabulary. You have run Qwen3.8-27B on a prompt and the final layer has handed you 248,320 raw numbers, one per output-tensor row. They are not probabilities: some are negative, some are large, and they certainly do not sum to anything convenient. Somebody now has to turn that pile into an actual distribution, and then somebody has to choose a token from it. Almost all of the probability you need for this course is the machinery for those two steps.

Start with what a distribution is. A probability is a number between zero and one expressing uncertainty under a model. For mutually exclusive possibilities the probabilities add to one. After any runtime masking, softmax produces a categorical distribution: non-negative values summing to one over the candidates the decoder may select. Qwen’s raw output vector has length 248,320, while its tokenizer artifacts map 248,077 ids. Notice what this makes explicit. The distribution is the model’s answer. Picking a token from it is a separate decision made by code you configure, not by the network.

Probability is always conditioned on something. We write $P(A)$ for the probability of $A$, and $P(A \mid B)$ for the probability of $A$ given that $B$ holds. Language modelling is conditional through and through: $P(x_t \mid x_1,\ldots,x_{t-1})$. The token Mercury is a fine continuation of the closest planet to the Sun is and a poor one after the liquid metal is. Conditioning is the whole game, and the history in lesson 0.2 is the history of how much of $x_1,\ldots,x_{t-1}$ a model could afford to look at.

The chain rule assembles a sequence probability from those conditionals:

This identity is why repeated next-token prediction defines a probability for a whole passage. It also explains an immediate practical problem: multiply a few hundred numbers below one and the result underflows to zero in floating point. So implementations work in logarithms. Because $\log(ab) = \log a + \log b$, the product becomes a sum. The log of a probability is at most zero, closer to zero for likelier events, tending to minus infinity for impossible ones. Training minimizes negative log-likelihood, which flips the sign so that better predictions give a smaller number, and averaging over tokens makes sequences of different length comparable.

Now the bridge from raw scores to that distribution. Given logits $z_i$, softmax computes

Exponentiating makes every value positive; dividing by the total makes them sum to one. One property is worth memorizing: adding the same constant to every logit leaves the probabilities unchanged, because the constant factors out of numerator and denominator. Softmax cares only about differences between logits.

Work the temperature example numerically, using three logits and ignoring the other 248,317 for legibility. Take $z = (4.0, 2.0, 1.0)$. Exponentiating gives $54.60$, $7.39$, and $2.72$, summing to $64.71$, so the probabilities are about $0.844$, $0.114$, and $0.042$. That is temperature 1.0, the preset Qwen3.8-27B uses in its thinking mode. Now apply temperature $T = 0.7$, the model’s instruct-mode preset. Temperature divides the logits before softmax: $z/T = (5.714, 2.857, 1.429)$. Exponentiating gives roughly $303.1$, $17.4$, and $4.17$, summing to $324.7$, so the probabilities become about $0.934$, $0.054$, and $0.013$. The leader gained nine points of probability and the tail lost most of what it had. Nothing about the model changed; only the sharpness of the reading.

There is a classic wrong turn here that is worth walking into deliberately. Suppose you had instead divided the probabilities by 0.7 and renormalized. You would get $0.844/0.7 = 1.206$, $0.163$, and $0.060$, summing to $1.429$; dividing through by that total returns $0.844$, $0.114$, $0.042$. Exactly the original distribution. Scaling every probability by a constant and renormalizing is a no-op, because the constant cancels in the same way an added constant cancels for logits. Temperature only does anything because it acts before the exponential, where a multiplicative change to the exponent becomes a nonlinear change to the result. If your sampler applies temperature to the wrong side of the softmax, the parameter will appear to do nothing at all, and this is a real bug people ship.

Two rules prevent most probabilistic reasoning errors. First, $P(A \mid B)$ is not $P(B \mid A)$; a symptom can be common among patients with a rare disease while the disease stays rare among people with the symptom. Second, expectation is not an outcome: the expected value $E[X] = \sum_x p(x) x$ of a fair die is 3.5, and no face shows 3.5. Expected loss, the quantity training minimizes, is an average over examples that no single example need resemble.

The working toolkit is small: normalize with softmax, condition on context, multiply conditionals with the chain rule, work in logs for stability, and keep the model’s distribution firmly distinct from the world it describes. With those five pieces, cross-entropy, entropy, perplexity, and every sampling method in track 7 are variations rather than new subjects.