Tensors

PyTorch tensors play the role that arrays played in the earlier chapters.

A tensor stores numbers and remembers facts about those numbers:

• shape
• dtype
• device
• whether gradients should be tracked

Shape is still the first thing to check. A tensor with shape (32, 128) can be read as 32 examples, each represented by 128 numbers, if that is the convention used by the model. PyTorch does not know the meaning of the axes. The reader must.

import torch

x = torch.tensor([[1.0, 2.0, 3.0],
                  [4.0, 5.0, 6.0]])

print(x.shape)  # torch.Size([2, 3])

This is not a new mathematical object. It is the same rectangular data you have already used, represented by a library that can also track devices and gradients.

The common mistake is to treat tensor operations as opaque commands. Keep reading them as shape transformations:

batch = torch.randn(64, 10)
weights = torch.randn(10, 3)
scores = batch @ weights

print(scores.shape)  # torch.Size([64, 3])

The multiplication works because the inner dimensions match. The result has one row per example and one score per output dimension.

The tensor name is new. The discipline is not: inspect shape before meaning.