LLM Primer

Batch and Feature Dimensions

Most training code processes more than one example at a time. That group is called a batch.

Suppose one example has 3 features:

[0.2, 1.5, -0.4]

A batch of 4 such examples can be written as a matrix:

[
  [ 0.2,  1.5, -0.4],
  [ 1.1, -0.3,  0.8],
  [-0.7,  0.0,  2.2],
  [ 0.4,  1.2,  1.9]
]
X: one batch of examplesx11x12x13x21x22x23x31x32x33x41x42x43batch axis4 examplesfeature axisshape = (4, 3)rows are examplescolumns are features
The batch dimension groups examples. The feature dimension describes each example.

The shape is (4, 3).

  • 4 is the batch size: how many examples are being processed together.
  • 3 is the feature size: how many numbers describe each example.

The batch dimension is not a new feature. It is packaging. It lets the same computation run for many examples at once.

Why the distinction matters

Imagine a layer that expects 3 input features and produces 2 output features. Its weight matrix has shape (3, 2).

If the input batch X has shape (4, 3), then:

XW has shape (4, 2)

The batch size 4 passes through. The feature size changes from 3 to 2.

This is the core pattern behind many neural layers:

(batch, input_features) x (input_features, output_features)
      -> (batch, output_features)
DL-C01-T02-001Exercise: Find the batch size

An input array has shape (12, 5), where rows are examples and columns are features. What is the batch size?

Compute it first, then check your number.

HintRows are examples

The batch dimension is the number of rows here.

SolutionWork it out

The shape (12, 5) means 12 examples, each with 5 features. The batch size is 12.

DL-C01-T02-002Exercise: Predict the output shape

Let X have shape (10, 4) and W have shape (4, 3). What is the output shape of XW?

Compute it first, then check your number.

HintUse the shared dimension

(10, 4) x (4, 3) is valid because the inner dimensions match.

SolutionWork it out

The inner dimension 4 is used in the multiplication. The outer dimensions remain, so the output shape is (10, 3).

DL-C01-T02-003Exercise: Spot a shape mismatch

A layer expects 5 input features. The input batch has shape (8, 4). Enter 1 if the layer can use this input directly, or 0 if the shapes do not match.

Compute it first, then check your number.

HintCompare feature counts

The batch size is not the problem. Check the feature dimension.

SolutionWork it out

The input shape (8, 4) means 8 examples with 4 features each. A layer expecting 5 input features cannot use those examples directly. The correct answer is 0.