Toy Regression and Classification

The same MLP structure can support different tasks.

For regression, the output may be one number:

prediction = hW2 + b2

Example: predict a house price from features.

For classification, the output may be one score per class:

scores = hW2 + b2

Example: choose among three classes.

The architecture can look similar. The interpretation of the output changes.

Shapes

Regression with one output:

H shape:  (batch, hidden_units)
W2 shape: (hidden_units, 1)
output:   (batch, 1)

Classification with three classes:

H shape:  (batch, hidden_units)
W2 shape: (hidden_units, 3)
scores:   (batch, 3)

Loss functions will decide how those outputs are judged. That comes later.

Exercise: Regression output shape

H has shape (10, 6) and the regression output layer uses W2 with shape (6, 1). What is the output shape?

Your answer

Compute it first, then check your number.

HintMultiply shapes

(10, 6) x (6, 1) gives the output.

SolutionWork it out

HW2 has shape (10, 1). The model produces one prediction for each of the 10 examples.

Exercise: Classification score shape

H has shape (10, 6) and a classifier has 4 classes. What should the score matrix shape be?

Your answer

Compute it first, then check your number.

HintOne score per class

The batch size stays 10.

SolutionWork it out

A classifier with 4 classes produces 4 scores per example. With 10 examples, the score matrix has shape (10, 4).