Summary and Revision Notes
Core vocabulary
| Word | Meaning | Example shape |
|---|---|---|
| scalar | one number | () |
| vector | one-axis array | (3) |
| matrix | two-axis array | (2, 3) |
| tensor | general array with axes | (4, 2, 3) |
| batch | group of examples processed together | first dimension in (batch, features) |
| feature | one measured or computed value describing an example | second dimension in (batch, features) |
| parameter | learned value in a model | weights W, bias b |
| activation | intermediate value produced inside a model | output of XW + b |
Shape pattern for a basic layer
If:
X has shape (batch, input_features)
W has shape (input_features, output_features)
then:
XW has shape (batch, output_features)
The batch size is carried through. The feature dimension changes.
Shape checks
- A reshape must preserve the total number of entries.
- A feature-wise bias should match the output feature size.
- Broadcasting reuses smaller arrays across compatible axes.
- Axis order is a convention. Name the axes before trusting the shape.
Common mistakes
- Treating the batch dimension as a feature.
- Forgetting that matrix multiplication needs matching inner dimensions.
- Reading a tensor shape without asking what each axis means.
- Assuming a framework will make the conceptual shape problem disappear.
- Flattening structured data without noticing what structure was lost.
Before moving on
You should be able to explain why (10, 4) x (4, 3) produces (10, 3), why (10, 4) x (5, 3) is not valid, and why (2, 3) can be flattened to (6) but not reshaped to (4, 2).