Review

Key ideas

• An MLP is a composition of affine maps and activations.
• A hidden layer sits between input and output.
• Hidden activations are intermediate values, not final predictions.
• Depth counts learned layers.
• Width counts units in a layer.
• Elementwise activations keep shape.
• Hidden representations are learned features used before prediction.
• Regression may output one number per example.
• Classification usually outputs one score per class.

Core formula

For a one-hidden-layer MLP:

H = relu(XW1 + b1)
scores = HW2 + b2

Common mistakes

• Counting the input size as a learned layer.
• Forgetting that activation changes values but not shape.
• Treating hidden activations as final predictions.
• Confusing hidden width with output class count.
• Trying to discuss training before defining how predictions are judged.

Before moving on

You should be able to trace the shapes in H = relu(XW1 + b1) and scores = HW2 + b2, and explain why the hidden representation comes before prediction.