Low-Rank Approximation
A low-rank approximation replaces a matrix with a simpler matrix that keeps its strongest structure.
If SVD gives:
then a rank-k approximation keeps only the top k singular values and their
directions.
The approximation has lower rank because it uses fewer independent directions than the original matrix.
Why This Works
Many datasets and matrices contain redundancy.
If most of the useful structure is concentrated in a few strong directions, a low-rank approximation can keep the main pattern while using fewer numbers.
This is the linear algebra behind many forms of compression.
For example, an image matrix may contain broad regions, edges, and shadows that can be captured by a smaller number of strong patterns. Keeping only those patterns gives a smaller representation. The image is not exact, but the main structure may remain visible.
What Gets Lost
Low-rank approximation discards weaker directions.
That can remove noise, but it can also remove small details that matter.
The important question is not whether low rank is always good. The question is whether the discarded directions are useful for the task.
If the weak directions are mostly noise, discarding them can help. If they contain rare but important detail, discarding them can harm the result.
So low-rank approximation is a controlled tradeoff. It exchanges exactness for simplicity. That tradeoff is useful only when the kept directions preserve what the task needs.
If a rank-3 approximation keeps the top singular directions, how many
directions are kept?
Compute it first, then check your number.
Hint
The rank number tells how many directions remain.
Solution
A rank-3 approximation keeps 3 directions. The rank number counts how many
independent singular directions remain in the approximation.
If singular values are 9, 4, 1, and 0.2, which two values does a rank-2
approximation keep?
Compute it first, then check your number.
Hint
Keep the largest two singular values.
Solution
A rank-2 approximation keeps the top two singular values: 9 and 4. The
smaller values, 1 and 0.2, are discarded because only two directions are
allowed to remain.
If the discarded singular values are very small, is the approximation often closer or farther from the original matrix?
Enter closer or farther.
Answer it first, then check.
Hint
Small discarded values mean little strength was removed.
Solution
It is often closer. Removing very weak directions changes the matrix less than removing strong directions.
Can a low-rank approximation remove important rare details?
Answer it first, then check.
Hint
Weak directions are not always useless for every task.
Solution
Yes. Low-rank approximation keeps the strongest directions, but a weak direction can still contain detail that matters for a specific task.
Enter 1 if low-rank approximation trades exact reconstruction for a simpler
matrix.
Compute it first, then check your number.
Hint
Ask what happens to directions that are not kept.
Solution
Enter 1. A low-rank approximation keeps fewer directions. That makes the
matrix simpler, but it usually gives up exact reconstruction.
Before Moving On
Low rank is useful when the strongest directions carry most of the signal.