Piecewise-Linear Behavior
ReLU is nonlinear, but it is made from linear pieces.
relu(x) = 0 when x <= 0
relu(x) = x when x > 0
Each side is simple. The bend at zero is what makes the whole function nonlinear.
A network with ReLU activations can combine many such bends. The result can approximate shapes that one straight boundary cannot express.
Why pieces matter
Think of a linear model as drawing one straight line. ReLU networks can build functions from many straight segments. More hidden units and layers can add more pieces.
This is not magic. It is composition: simple operations arranged so their bends combine.
For x = -2, which ReLU piece applies? Enter 0 for the flat piece relu(x) = 0, or 1 for the identity piece relu(x) = x.
Compute it first, then check your number.
HintCompare with zero
Negative values are on the left side of the ReLU bend.
SolutionWork it out
Since -2 <= 0, ReLU uses the flat piece and returns 0.
Enter 1 if the bend at zero makes ReLU nonlinear overall, or 0 if ReLU is one linear function everywhere.
Compute it first, then check your number.
HintLook at the bend
One straight formula cannot describe both sides as one linear map.
SolutionWork it out
ReLU is 0 on one side and x on the other. The bend at zero makes it
nonlinear as a whole.