Shape Notation and Axis Conventions

A tensor shape is usually written as a tuple of axis sizes.

(32, 10)
(8, 28, 28)
(4, 12, 64)

The numbers alone are not enough. A shape becomes meaningful only when you know what each axis represents.

(32, 10)       32 examples, 10 features
(8, 28, 28)    8 images, each 28 by 28
(4, 12, 64)    4 sequences, 12 positions, 64 features per position

Different libraries and domains sometimes use different axis orders. For example, image data may be written as:

(batch, height, width, channels)

or:

(batch, channels, height, width)

Neither order is morally better. What matters is consistency.

Batch-first convention

In this course, early chapters usually use batch-first shapes:

(batch, features)
(batch, height, width)
(batch, sequence_length, features)

This convention keeps the first question simple: how many examples are moving through the model together?

Later, we will point out when a different convention appears.