Review

Key ideas

• A loss function compares prediction with target.
• Signed errors can cancel.
• Mean squared error averages squared errors.
• Binary cross-entropy is used for binary classification probabilities.
• Multiclass cross-entropy uses the probability assigned to the true class.
• Logits are raw scores, not probabilities.
• Softmax turns logits into probabilities that sum to one.
• Stable softmax subtracts the maximum logit before exponentiating.
• Loss is a training signal, not the full goal.

Common formulas

error = prediction - target
squared_error = error^2
MSE = average squared error
cross_entropy = -log(p_true)

Common mistakes

• Averaging signed errors and thinking zero means good predictions.
• Treating logits as probabilities.
• Forgetting that cross-entropy looks only at the true-class probability for one example.
• Treating the loss as the whole product goal.

Before moving on

You should be able to compute simple MSE examples, identify p_true for cross-entropy, separate logits from probabilities, and explain why stable softmax subtracts the maximum logit.