Batches and Shape Reasoning

Machine learning rarely processes one example at a time.

A batch groups many examples together. If each example is a vector, a batch can be stored as a matrix.

For example:

X has shape batch_size x features

If there are 4 examples and each example has 3 features, then:

X has shape 4 x 3

Read this as:

4 examples, 3 features per example

This chapter uses the common layout:

rows = examples
columns = features

Some libraries or papers may arrange axes differently. The habit is not to memorize one layout forever. The habit is to name what each axis means before you multiply.

MATH-C03-T07-001Exercise: Read a batch shape

A batch matrix $X$ has shape 8 x 5. How many examples are in the batch?

Your answer

Compute it first, then check your number.

HintExamples first

In batch_size x features, the first number is the number of examples.

SolutionBatch size

The shape 8 x 5 means 8 examples and 5 features per example. In this chapter's convention, the first axis is the batch axis.

MATH-C03-T07-005Exercise: Name the axes

In this chapter's batch convention, enter 1 if the rows of a batch matrix represent examples.

Your answer

Compute it first, then check your number.

HintRead the convention

We are using batch_size x features.

SolutionRows as examples

Enter 1. In this chapter, a batch matrix is read as examples by features: rows are examples, columns are features.

Batch Through A Layer

Suppose:

X has shape 4 x 3
W has shape 3 x 2

Then:

XW has shape 4 x 2

The 3 feature dimension matches. The output keeps the batch size 4 and produces 2 output features.

MATH-C03-T07-002Exercise: Predict a batch output shape

If $X$ has shape 8 x 5 and $W$ has shape 5 x 3, what shape does $XW$ have?

Your answer

Compute it first, then check your number.

HintKeep the outside dimensions

The middle 5 values match. The output keeps 8 and 3.

SolutionOutside dimensions

(8 \times 5)(5 \times 3) \rightarrow 8 \times 3

The two 5 dimensions match and disappear into the dot products. The outside dimensions, 8 examples and 3 output features, remain.

Why This Matters

Shape reasoning prevents many mistakes before code runs.

If a layer expects 3 input features, the input must end with size 3. If it produces 2 output features, the result ends with size 2.

In code, this often means checking the last dimension of the input against the first dimension of the weight matrix.

MATH-C03-T07-003Exercise: Spot a layer mismatch

Enter 1 if an input batch with shape 4 x 3 can multiply a weight matrix with shape 5 x 2, or 0 if it cannot.

Your answer

Compute it first, then check your number.

HintMiddle dimensions

Matrix multiplication needs the middle dimensions to match.

SolutionShape mismatch

(4 \times 3)(5 \times 2)

The middle dimensions are 3 and 5, so the multiplication is not valid. The input examples have only 3 features, but the weight matrix expects 5 input features.

What The Output Rows Mean

Each output row corresponds to one input example.

If $X$ has 8 examples and $W$ produces 3 features, then $XW$ has 8 rows and 3 columns.

The batch dimension is carried through the layer.

MATH-C03-T07-004Exercise: Track the batch dimension

If $X$ has shape 10 x 4 and $W$ has shape 4 x 6, how many rows does $XW$ have?

Your answer

Compute it first, then check your number.

HintRows are examples

The number of examples remains 10.

SolutionBatch preserved

(10 \times 4)(4 \times 6) \rightarrow 10 \times 6

The output has 10 rows. Each input example produces one output row, so the batch size is preserved.

Next, we interpret a matrix as a transformation.