Parameters Versus Data

Deep learning becomes clearer when data and parameters do not blur together.

Data comes from outside the model.

Parameters are stored inside the model and adjusted by training.

For a training example:

x = input features
y = target value

For a model:

w, b = parameters

The model uses x, w, and b to compute a prediction:

y_hat = wx + b

Then the prediction is compared with the target y.

The target tells the training process what the model should have predicted. The target is not a parameter either.

What changes during training?

During training, the dataset is read many times. The examples do not become model parameters. Instead, the parameters are adjusted so the model computes better predictions on examples like the ones it saw.

This distinction matters because memorizing data is not the same as learning a useful function.