Loss Functions

A loss function turns predictions and targets into a scalar training signal.

PyTorch provides common losses in torch.nn, but their role is the same as in the earlier chapters.

import torch
from torch import nn

pred = torch.tensor([2.0, 4.0, 6.0])
target = torch.tensor([1.0, 5.0, 7.0])

loss_fn = nn.MSELoss()
loss = loss_fn(pred, target)

For mean squared error, the visible computation is:

(2 - 1)^2 = 1
(4 - 5)^2 = 1
(6 - 7)^2 = 1
mean = 1

The PyTorch call is shorter, but it compresses the same arithmetic.

Loss functions are usually chosen to match the modeling task:

• squared error for many regression tasks
• cross-entropy for many classification tasks
• task-specific losses when the training objective has a special structure

The loss must become a scalar before backward() is called. That scalar answers one question: how bad was this batch under the current parameters?

The loss is the bridge from model output to parameter change.