Get ready for the GARP Risk and AI Exam with flashcards and multiple choice questions. Each question comes with hints and explanations. Prepare for success!

Multiple Choice

Which concept refers to focusing on a single metric rather than a broader view of model performance?

Focusing on a single metric or point estimate of model performance while ignoring other facets of how the model behaves is a common pitfall. In practice, evaluating a model should go beyond one number—like accuracy or AUC—to include multiple performance aspects such as calibration, robustness to data shifts, fairness, and how confident we are in the estimates. A single metric can be volatile or masking weaknesses in other areas; for instance, a model might score well on overall accuracy but perform poorly on minority groups or miscalibrate its probability estimates. By considering a broader set of metrics and the uncertainty around them, you get a fuller, more reliable view of how the model will perform in real-world settings. Anchoring, confirmation bias, and model selection bias describe different biases in information handling, belief, or evaluation setup, rather than the specific issue of fixating on one point estimate.

Focusing on a single metric or point estimate of model performance while ignoring other facets of how the model behaves is a common pitfall. In practice, evaluating a model should go beyond one number—like accuracy or AUC—to include multiple performance aspects such as calibration, robustness to data shifts, fairness, and how confident we are in the estimates. A single metric can be volatile or masking weaknesses in other areas; for instance, a model might score well on overall accuracy but perform poorly on minority groups or miscalibrate its probability estimates. By considering a broader set of metrics and the uncertainty around them, you get a fuller, more reliable view of how the model will perform in real-world settings. Anchoring, confirmation bias, and model selection bias describe different biases in information handling, belief, or evaluation setup, rather than the specific issue of fixating on one point estimate.