Confusion Matrix

2.7 — Confusion Matrix

Recall first. For a disease screen, which is more dangerous: a false positive or a false negative? The answer depends on the action and cost—state a reason before reading.

Four outcomes, not one accuracy number

For binary classification, compare predicted class with the true class:

The confusion matrix is the count table [ [TN, FP], [FN, TP] ] under the common negative-first convention. Always label axes because libraries and textbooks may order classes differently. scikit-learn defines its matrix with rows as true classes and columns as predicted classes.¹

From it:

N = TP + TN + FP + FN
accuracy = (TP + TN)/N
error rate = (FP + FN)/N

Accuracy is useful when classes and error costs are reasonably balanced. It can be misleading for rare positives: predicting every case negative may be highly accurate but useless.

Worked calculation

Suppose a detector gives TP=36, FN=4, FP=10, TN=50. Total is 100 and the matrix is:

              predicted +   predicted −
actual +          36             4
actual −          10            50

Accuracy is (36+50)/100=0.86, or 86%; error rate is 14%. There are 40 actual positives and 60 actual negatives. This same table will support sensitivity, specificity, precision, recall, F1, and kappa in the next topics.

Why the matrix is a diagnostic tool

The four cells reveal which mistakes occur. A threshold change can move cases between TP/FN and FP/TN, changing the operating point without refitting the model. In multiclass classification, the matrix becomes K×K: diagonal counts are correct; off-diagonal cells show which classes are confused.

Counts must come from an honest held-out or cross-validated prediction. A confusion matrix on training predictions can hide overfitting. For imbalanced data, report the class distribution and several complementary metrics rather than selecting accuracy alone.²

Exercise

A test has 200 cases: TP=18, FN=2, FP=30, TN=150. Find accuracy and the number of actual positives and negatives.

Exam lens

Draw the 2×2 table first, define every abbreviation, then substitute counts. Mention that row/column orientation must be stated and that accuracy alone can fail under imbalance.

Rapid revision checklist

• Can I place TP, TN, FP, and FN correctly?
• Can I calculate accuracy and error rate?
• Can I identify which cell represents a missed positive?
• Can I explain why a confusion matrix is more informative than accuracy?

Key takeaways

• The matrix counts correct and incorrect predictions by actual and predicted class.
• TP/TN are correct; FP/FN are errors, with the positive class defining the names.
• Accuracy is (TP+TN)/N, but class imbalance and unequal costs require more metrics.
• State matrix orientation before calculating.

Sources