Requires Grad and Autograd

Autograd is PyTorch's automatic differentiation system.

When a tensor is created with requires_grad=True, PyTorch records operations that use it. After a scalar loss is computed, calling backward() sends derivatives backward through the recorded computation graph.

import torch

w = torch.tensor(2.0, requires_grad=True)
x = torch.tensor(3.0)

y = w * x
loss = (y - 10.0) ** 2
loss.backward()

print(w.grad)

The important part is not the syntax. The important part is the contract:

• the forward pass builds a computation
• the loss is a scalar
• backward() computes derivatives of that scalar
• gradients are stored on tensors that requested them

Autograd does not remove the chain rule. It applies the chain rule over the graph made by the operations.

This also explains why not every tensor needs gradients. Input data often does not need a gradient. Trainable weights usually do. Intermediate tensors may be part of the graph, but they are not always where gradients are stored for later use.

Think of autograd as a careful assistant for the derivative bookkeeping, not as a substitute for knowing what gradients mean.