Which statistical measure is used to compare and select models, especially in regression, where a lower number indicates a better model?

• A. Akaike Information Criterion (AIC)
• B. Bayesian Information Criterion (BIC)
• C. Adjusted R-Squared
• D. Mean Squared Error

Answer: (A)

Comparing models by balancing fit with complexity is the key idea here. Akaike Information Criterion (AIC) does exactly that: it rewards how well the model fits the data (through the likelihood) but penalties adding more parameters to avoid overfitting. The formula AIC = 2k − 2 ln(L) captures this trade-off, where k is the number of estimated parameters and L is the likelihood of the model given the data. Lower AIC means a better balance, so you pick the model with the smallest AIC.

Bayesian Information Criterion behaves similarly but imposes a stronger penalty for complexity, especially with larger sample sizes. Adjusted R-squared adjusts explained variance for the number of predictors but isn’t a likelihood-based model comparison criterion. Mean Squared Error focuses on predictive error but doesn’t penalize complexity, which can lead to overfitting if used alone.