Which statistical measure is used to compare and select models, especially in regression, where a lower number indicates a better model?

Which statistical measure is used to compare and select models, especially in regression, where a lower number indicates a better model?

• A. Akaike Information Criterion (AIC)
• B. Bayesian Information Criterion (BIC)
• C. Adjusted R-Squared
• D. Mean Squared Error

Answer: (A)

Comparing models by balancing fit with complexity is the key idea here. Akaike Information Criterion (AIC) does exactly that: it rewards how well the model fits the data (through the likelihood) but penalties adding more parameters to avoid overfitting. The formula AIC = 2k − 2 ln(L) captures this trade-off, where k is the number of estimated parameters and L is the likelihood of the model given the data. Lower AIC means a better balance, so you pick the model with the smallest AIC. Bayesian Information Criterion behaves similarly but imposes a stronger penalty for complexity, especially with larger sample sizes. Adjusted R-squared adjusts explained variance for the number of predictors but isn’t a likelihood-based model comparison criterion. Mean Squared Error focuses on predictive error but doesn’t penalize complexity, which can lead to overfitting if used alone.

Comparing models by balancing fit with complexity is the key idea here. Akaike Information Criterion (AIC) does exactly that: it rewards how well the model fits the data (through the likelihood) but penalties adding more parameters to avoid overfitting. The formula AIC = 2k − 2 ln(L) captures this trade-off, where k is the number of estimated parameters and L is the likelihood of the model given the data. Lower AIC means a better balance, so you pick the model with the smallest AIC.

Bayesian Information Criterion behaves similarly but imposes a stronger penalty for complexity, especially with larger sample sizes. Adjusted R-squared adjusts explained variance for the number of predictors but isn’t a likelihood-based model comparison criterion. Mean Squared Error focuses on predictive error but doesn’t penalize complexity, which can lead to overfitting if used alone.