PadhaikoAll subjects

Past exam question: 2081 Chaitra (Regular), Q2(b)(ii)

Foundation of Data Science · 2 marks

Open in the Padhaiko app

Topic: Model Performance Measures: Confusion Matrix, ROC & Clustering

Question

Considering the test data size as 50,000, a typical classification model experimentation results are as shown on the confusion matrix.

PREDICTED CLASS: YesPREDICTED CLASS: No
ACTUAL CLASS: Yes10001000
ACTUAL CLASS: No100047000

Explain the difference on resulted accuracy and -score value that you have calculated.

Solution

The Accuracy Paradox Under Severe Class Imbalance: The confusion matrix exhibits extreme class imbalance:

Because the majority class overwhelmingly dominates the sample, the high accuracy of is almost entirely driven by the True Negatives. A trivial, baseline model that blindly predicts "No" for all inputs would achieve an identical accuracy (), while providing zero predictive utility for detecting the positive class. Thus, accuracy creates a false impression of superior performance.

How F1-Score Reflects Actual Minority-Class Performance: Unlike accuracy, the -score completely disregards True Negatives () and focuses strictly on the positive class ("Yes"):

The -score penalizes extreme imbalances between precision and recall via their harmonic mean, producing . This accurately exposes that the model is only moderately effective () on the minority class of interest, demonstrating why -score is far more informative than accuracy in imbalanced classification settings.

Answer:

The large discrepancy (Accuracy vs. ) is caused by severe class imbalance ( negative vs. positive). Accuracy is dominated by the large number of True Negatives (), masking poor positive-class detection, whereas the -score ignores True Negatives and evaluates precision and recall on the positive minority class, correctly revealing moderate performance ().

All past exam questions for Foundation of Data Science