Why Linear Models Are Limited
Linear models are useful because they are simple, inspectable, and fast.
They are limited because their decision boundaries are linear.
In two dimensions, a linear classifier draws a line. In three dimensions, it draws a plane. In higher dimensions, it draws a hyperplane.
That is powerful when the data can be separated by one straight boundary. It is not enough when the useful pattern bends, loops, or depends on feature interactions that the linear model cannot express.
A simple limitation
Suppose a task needs this rule:
positive if both features are high or both features are low
negative otherwise
One straight line cannot separate that pattern in the original two-dimensional input space.
This is one reason neural networks add nonlinear activations and multiple layers. The next chapters build toward that idea carefully.
In two input dimensions, what shape is the decision boundary of a binary linear classifier? Enter 1 for line, 2 for circle, or 3 for curve.
Compute it first, then check your number.
HintLinear in two dimensions
A linear equation in two variables forms a line.
SolutionWork it out
The boundary has the form x . w + b = 0. In two dimensions, that is a
line.
Enter 1 if nonlinear activations help neural networks represent patterns that one linear boundary cannot, or 0 if they have no effect on the model family.
Compute it first, then check your number.
HintThink about capacity
Linear maps alone stay limited to linear transformations.
SolutionWork it out
Nonlinear activations let networks build richer functions from simple pieces. Without them, stacked linear maps collapse back into another linear map.