Model Assessment: AIC, BIC, and Adjusted R-Squared

AIC (Akaike Information Criterion)

AIC = -2 * ln(L) + 2k, where L is the maximized likelihood and k is the number of estimated parameters. AIC balances goodness of fit (first term) with model complexity (second term). Lower AIC indicates a better model. AIC is asymptotically equivalent to leave-one-out cross-validation for linear models. It tends to select models that predict well but may include unnecessary parameters.

For linear regression with normally distributed errors: AIC = n * ln(SSE/n) + 2k + constant. The constant does not affect model comparisons.

BIC (Bayesian Information Criterion)

BIC = -2 * ln(L) + k * ln(n), where n is the sample size. BIC penalizes complexity more heavily than AIC (since ln(n) > 2 for n >= 8). As a result, BIC tends to select simpler models than AIC. BIC is consistent: as n increases, it selects the true model with probability approaching 1 (assuming the true model is among the candidates). AIC is not consistent; it tends to overfit asymptotically.

For model selection, compute AIC and BIC for each candidate model and choose the model with the smallest value of the chosen criterion.

Adjusted R-Squared

Adjusted R^2 = 1 - (SSE/(n-p-1))/(SST/(n-1)), where p is the number of predictors. Unlike R^2, it penalizes for additional predictors and can decrease. Adjusted R^2 is simpler to compute than AIC/BIC but only applies to linear regression with normally distributed errors. For Exam SRM, AIC is generally preferred for prediction-focused model selection, BIC for identifying the true model, and adjusted R^2 as a quick comparison tool for linear models with the same response variable.