The feed-forward block & where knowledge lives

1

Concept

Attention receives most of the Transformer’s publicity because it creates visible links between positions. Yet a large fraction of a dense Transformer’s parameters and computation lives in the feed-forward network (FFN), also called the multilayer perceptron or MLP block. Attention routes information; the FFN transforms it.

The original block applies two linear maps with a nonlinearity between them:

The first projection expands from model width $d_{model}$ to a larger intermediate width $d_{ff}$. The second projects back to $d_{model}$ so the result can rejoin the residual stream. The same weights are used at every sequence position, and positions can be processed in parallel. There is no token-to-token mixing inside this ordinary FFN.

Expansion matters because a wide intermediate space can represent many feature detectors. A row of $W_1$ responds strongly when the incoming residual vector aligns with its learned direction. The activation function suppresses or reshapes that response. Columns of $W_2$ then write combinations of activated features back into the model space. This resembles a learned pattern bank more than a small generic “extra layer.”

Transformer variants use ReLU, GELU, or gated activations such as SwiGLU. A gated FFN forms two projections: one produces candidate features and another controls their strength, followed by element-wise multiplication and an output projection. Parameter budgets are often adjusted when comparing gated and ungated blocks, so simply quoting intermediate width can misrepresent capacity.

Research has described FFNs as key–value memories: input directions behave like keys that recognize patterns, while output directions write associated information. This is a productive lens. A hidden activation can respond to a textual pattern and add a feature useful for a likely continuation. It helps explain why FFN interventions can change factual behavior.

But “facts live in the FFN” is too literal. Model behavior emerges from the whole computation. Token embeddings establish features; attention retrieves contextual signals; residual connections superpose updates; normalization changes effective scale; several FFNs compose across depth; and the output projection reads the final state. A factual association may depend on a route through many components. Individual neurons are not guaranteed database rows with stable natural-language labels.

Consider the token “Paris” after attention has gathered features indicating the pattern “capital of France is.” The FFN at that position may detect the conjunction and write a feature that the language-model head associates with a likely next token. In a different context, “Paris” arrives with different residual features and activates a different subset of the same shared FFN.

The block is expensive because the wide projections touch every token. For a fixed sequence length, its arithmetic can rival or exceed attention, especially when context is not very long. Sparse mixture-of-experts architectures later replace one dense FFN with a router and several expert FFNs, activating only a subset per token. That changes parameter capacity and communication while preserving the basic transform-at-each-position role.

When inspecting an FFN, track both shapes and residual effects. Ask which directions activate, what they write, how large the update is relative to the existing stream, and whether later layers preserve it. Activation alone is not causation; ablation or controlled intervention gives stronger evidence.

The stable mental model is alternating communication and computation. Attention brings relevant context to each address. The FFN expands that local state, applies nonlinear feature selection, and compresses an update back into the shared residual stream. Both halves are essential to a Transformer block.