Broadcasting

Broadcasting lets NumPy combine arrays with compatible but different shapes.

The simplest case is scalar broadcasting:

NumPy acts as if 10 were available for every position.

Add one vector to every row

X.shape is (2, 3). bias.shape is (3,).

NumPy adds the bias vector to each row.

This pattern appears constantly in neural networks: add one bias vector to a batch of row vectors.

Compare dimensions from the right

Read shapes from the right.

(2, 3)
   (3,)

The last dimensions match: 3 and 3. The missing leading dimension in bias can be repeated across the two rows.

More generally, two compared dimensions are compatible when:

• they are equal; or
• one of them is 1.

A missing leading dimension also behaves like 1. NumPy expands the size-1 or missing dimension to match; it does not copy an incompatible dimension into place.

For example:

(2, 3)
(2, 1)

are compatible. The second array supplies one value for each row, and that value is used across the row's three columns. By contrast, (2, 3) and (2,) are incompatible: comparing from the right gives 3 and 2, and neither is 1.

Start with this practical habit: write the shapes under each other and compare from the right.

Broadcasting can hide mistakes

Broadcasting is powerful, but it can also let wrong code run. If the result shape surprises you, stop and inspect.