Summary and Revision Notes
Key ideas
- A model is a function with parameters.
- Inputs are supplied by data.
- Parameters are stored by the model and changed by training.
- Outputs are computed predictions.
- A forward pass computes predictions using current parameters.
- Prediction happens before loss, gradient, and update.
- An architecture defines a family of possible functions.
- Parameters choose one function from that family.
Notation
| Notation | Meaning |
|---|---|
x | input |
y | target |
y_hat | prediction |
theta | model parameters |
f(x; theta) | model output for input x using parameters theta |
Common mistakes
- Treating input data as if it were learned by the model.
- Forgetting that targets are supplied by the dataset.
- Saying the model learns during the forward pass.
- Talking about capacity without saying what function family the architecture can represent.
Before moving on
You should be able to compute y_hat = wx + b, name which values are data and which are parameters, and explain why training must compute predictions before it can improve them.