Model Comparison and Information Criteria for MAS-I

Information Criteria

Information criteria balance model fit against complexity. AIC = -2*ln(L) + 2k, where L is the maximized likelihood and k is the number of parameters. BIC = -2*ln(L) + k*ln(n) penalizes complexity more heavily for large samples. Lower values indicate better models. AIC is asymptotically equivalent to leave-one-out cross-validation, while BIC is consistent (selects the true model as n grows, if it is among candidates). AICc = AIC + 2k(k+1)/(n-k-1) corrects for small samples. For GLMs, the deviance replaces -2*ln(L) in relative comparisons.

Cross-Validation and Other Methods

K-fold cross-validation splits data into K subsets, trains on K-1, and tests on the held-out fold, rotating through all K folds. The average test error estimates out-of-sample performance. Leave-one-out (LOO) is a special case with K=n. Likelihood ratio tests compare nested models: the statistic -2*(ln(L_reduced) minus ln(L_full)) follows chi-squared with degrees of freedom equal to the difference in parameters. For non-nested models, Vuong's test provides a formal comparison. Exam MAS-I tests the application of these methods to select among competing actuarial models.