ROC Curve

2.11 — ROC Curve

Recall first. A classifier outputs scores, not just labels. What changes when the decision threshold is lowered: TP, FP, or both? Predict the direction before reading.

One curve over all thresholds

The receiver operating characteristic (ROC) curve plots:

TPR = sensitivity = TP/(TP+FN)
FPR = FP/(FP+TN) = 1 − specificity

for many classification thresholds. Each threshold gives one confusion matrix and one (FPR, TPR) point. Lowering the threshold generally labels more examples positive, increasing both TPR and FPR; the exact path depends on the score distributions. A useful model ranks positives above negatives, producing a curve toward the top-left.¹²

The diagonal from (0,0) to (1,1) is random ranking in expectation. A perfect classifier reaches (0,1). ROC is threshold-independent as a curve but any deployed operating point is threshold-dependent.

AUC interpretation

The area under the ROC curve (ROC-AUC) summarizes ranking ability: under standard assumptions, it equals the probability that a randomly selected positive receives a higher score than a randomly selected negative (with ties handled by convention). 0.5 is random-ranking performance; 1 is perfect ranking. AUC does not tell you the threshold, calibration, prevalence, or action cost.

For highly imbalanced applications, a precision–recall curve can be more informative about positive-alert quality because ROC’s FPR denominator is all negatives, which may make many false positives look small when negatives are enormous. Choose the curve that matches the decision question.¹

Worked threshold trace

Suppose four cases have scores:

At threshold >0.85, predictions are +,-,-,-: TP=1,FN=1,FP=0,TN=2, so (FPR,TPR)=(0,0.5). At threshold >0.65, predictions are +,+,+,-: TP=2,FN=0,FP=1,TN=1, so (0.5,1). Lowering the threshold moved up and right. A threshold between 0.80 and 0.70 cannot separate the second positive from the 0.80 negative; ranking quality limits the curve.

Exercise

A threshold gives TP=45, FN=5, FP=20, TN=130. Compute the ROC point. If the threshold is lowered and 3 FN become TP while 10 TN become FP, compute the new point.

Exam lens

Label axes FPR and TPR, explain each threshold produces one point, state diagonal/random and top-left/perfect, then mention AUC’s ranking interpretation and limitations. Do not confuse FPR with false-discovery rate FP/(TP+FP).

Rapid revision checklist

• Can I write TPR and FPR formulas?
• Can I explain why threshold changes trace a curve?
• Can I calculate a ROC point from TP/FN/FP/TN?
• Can I state what AUC does and does not measure?
• Can I explain why PR curves may help with rare positives?

Key takeaways

• ROC plots sensitivity against false-positive rate across thresholds.
• AUC summarizes ranking, not a chosen operating threshold or calibration.
• Lowering a threshold usually increases both TPR and FPR.
• ROC must be interpreted with prevalence, costs, and possibly precision–recall behavior.

Sources