llmdeepdive
PT

Type to search

    Learning track

    Neural Network Fundamentals

    Learn how neural networks represent functions, measure error, propagate gradients, and improve through optimization.

    03

    foundations · 9 available lessons

    Neural Network Fundamentals

    Learn how neural networks represent functions, measure error, propagate gradients, and improve through optimization.

    1. 03.01The perceptron
    2. 03.02Multilayer perceptrons and non-linearity
    3. 03.03Activation functions: ReLU, GELU, and SwiGLU
    4. 03.04Loss functions and cross-entropy
    5. 03.05Backpropagation, visualized
    6. 03.06Gradient descent and the loss landscape
    7. 03.07Optimizers: SGD to Momentum to Adam to AdamW
    8. 03.08Initialization, normalization, and residuals
    9. 03.09Overfitting, regularization, and the bitter lesson