Introduction

The earlier chapters gave us three readings of the same objects.

A vector can be a point, a direction, or a row of measured features. A matrix can be a table, a product rule, or a map from one space to another. Geometry tells us what those objects are doing.

This chapter asks the next question:

Which outputs are possible, and what structure explains them?

Linear systems ask which vectors can be made from other vectors. Decompositions ask what simpler pieces are hidden inside a matrix.

Together, they explain ideas that appear later in machine learning:

• whether a set of features contains repeated information
• when a target can be matched exactly
• which directions in data matter most
• why compression can work
• how a matrix can stretch, rotate, collapse, or preserve parts of a space
• why lower-dimensional structure often appears inside high-dimensional data

The Main Question

The central question of this chapter is:

What does this matrix let us reach, preserve, lose, or emphasize?

That question is more useful than memorizing many procedures. A procedure tells you how to compute. The question tells you what the computation means.

For example, in a system like:

the vector b is the target. The matrix A describes the available directions or measurements. The unknown vector x asks for weights that make the target.

This is the chapter where equations begin to look like geometry.

There is a second habit too. When a matrix cannot reach every target, do not only ask for a formula. Ask what was missing.

Did the columns fail to span the right space? Did some input direction disappear? Did most of the useful signal live in only a few directions? Those questions are the bridge from computation to understanding.

What This Chapter Covers

This chapter introduces:

• linear systems
• span and linear combinations
• linear independence and basis
• rank and dimension
• column space and null space
• eigenvalues and eigenvectors
• diagonalization
• singular value decomposition
• low-rank approximation
• PCA

The goal is working understanding. We will use small examples and geometric meaning before formal algorithms.

What We Will Not Do Yet

We will not turn this chapter into a full numerical linear algebra course. Later work can cover stable algorithms, large sparse matrices, and implementation details. Here, the goal is to understand what the objects mean and why they keep appearing in ML.

Before Moving On

You should be comfortable with matrix-vector products, dot products, projection, and the idea that a matrix maps input vectors to output vectors.