Multiclass Cross-Entropy
Multiclass cross-entropy is used when one example belongs to one of several classes.
If the true class has predicted probability p_true, then:
loss = -log(p_true)
The formula is short because only the probability assigned to the true class matters for that example.
Example
Suppose the model predicts:
probabilities = [0.1, 0.7, 0.2]
If the true class is class 2, then:
p_true = 0.7
loss = -log(0.7)
If the true class is class 1, then:
p_true = 0.1
loss = -log(0.1)
The second case has higher loss because the model assigned low probability to the true class.
Probabilities are [0.2, 0.5, 0.3]. Using one-based class numbering, the true class is class 2. What is p_true?
Compute it first, then check your number.
HintUse the true class index
Class 2 means the second entry.
SolutionWork it out
The true class is class 2, so p_true is the second probability: 0.5.
Which true-class probability gives lower cross-entropy loss: 0.8 or 0.3? Enter the lower-loss probability.
Compute it first, then check your number.
HintCross-entropy is -log(p_true)
-log(p) is smaller when p is larger.
SolutionWork it out
Cross-entropy is -log(p_true). A larger p_true gives a smaller loss, so
0.8 is better than 0.3.