Shape Debugging

Shape debugging is the practice of finding where the expected tensor shape stops matching the actual tensor shape.

It is one of the most useful habits in deep learning.

Write the expected shape first

Before writing or reading a layer, write the shape you expect.

X: (batch, input_features)
W: (input_features, output_features)
b: (output_features)
output: (batch, output_features)

Then compare each real tensor to that expectation.

Check the first wrong place

When a later operation fails, the cause may be earlier. Do not only inspect the line that produced the error. Trace the shapes from the input forward.

input -> reshape -> layer -> activation -> loss

At each step, ask:

• What shape did I expect?
• What shape did I get?
• Which axis changed?
• Was that change intended?

Common shape mistakes

The most common mistakes are simple:

• using feature count where batch size belongs;
• flattening too early;
• forgetting that a bias follows output features;
• using a weight matrix with the wrong input feature size;
• mixing two axis conventions without noticing.

Shape debugging is not separate from understanding. It is understanding made testable.