Review

Key ideas

• Training, validation, and test splits have different roles.
• Metrics must match the task and the cost of mistakes.
• Learning curves reveal behavior that final metrics hide.
• Gradient norms summarize update signal size.
• Activation statistics show whether information is flowing.
• Parameter histograms show whether weights are plausible or suspicious.
• Dead ReLU detection tracks silent units or layers.
• Sanity checks test whether the setup can learn simple cases.
• Multiple seeds give better evidence than one lucky run.

Common formulas

accuracy = correct / total
mean_absolute_error = sum(abs(error)) / count
gradient_l2_norm = sqrt(sum(gradient_i^2))
zero_fraction = zero_count / total_count

Common mistakes

• Reporting only final accuracy.
• Tuning repeatedly on the test set.
• Ignoring learning curves and internal statistics.
• Treating one random seed as decisive evidence.
• Debugging architecture before checking data, labels, shapes, and loss.

Before moving on

You should be able to choose a basic metric, read a learning curve, compute a gradient norm, compute a zero fraction, and explain why repeated evidence is stronger than one run.