Exercise solutions

DL-C17-C-001: The tensor has 16 * 10 = 160 scalar entries.

DL-C17-C-002: The weight matrix has 3 * 10 = 30 entries. The bias has 3 entries. The total is 30 + 3 = 33.

DL-C17-C-003: The SGD update is:

4.0 - 0.1 * 1.5 = 4.0 - 0.15 = 3.85

DL-C17-C-004: There are 96 / 24 = 4 full batches.

DL-C17-C-005: The correct answer is 2. loss.backward() computes gradients. The optimizer update happens when optimizer.step() is called.

DL-C17-C-006: The correct answer is 1. model.parameters() returns the trainable tensors that the optimizer updates.

DL-C17-C-007: The correct answer is 1. A state dict stores learned tensor values such as weights and biases.

Written practice solutions

A five-line training step can be read as: clear old gradients, make predictions for a batch, compute the loss, compute gradients by backpropagation, and update the parameters.

zero_grad() appears before backward() because gradients accumulate by default. Clearing them makes the current step depend on the current batch rather than on leftover gradient values.

A module is the container that defines computation and owns trainable state. A parameter is one of the trainable tensors inside that state.

PyTorch comes after the earlier chapters because it compresses concepts that must first be visible: tensors, forward passes, losses, gradients, updates, and batches.