Machine Learning for Insurance Pricing: Gradient Boosting

Gradient Boosting Fundamentals

Gradient boosting builds an ensemble of weak learners (typically decision trees) sequentially, with each new tree fitting the residuals (negative gradient of the loss function) from the current model. Starting from an initial prediction F_0(x) = argmin sum of L(y_i, gamma), each iteration adds a tree h_m(x) multiplied by a learning rate nu: F_m(x) = F_{m-1}(x) + nu*h_m(x). For squared error loss, residuals are simply y_i minus F_{m-1}(x_i). For Poisson deviance (used in frequency modeling), the pseudo-residuals involve the log link. Smaller learning rates with more iterations generally improve generalization.

Application to Insurance

In insurance pricing, gradient boosting machines (GBMs) handle nonlinear effects and interactions automatically without pre-specification. Key hyperparameters include tree depth (controls interaction order), number of trees, learning rate, and subsampling fraction. Variable importance measures rank predictor contributions. Partial dependence plots reveal the marginal effect of each variable. Compared to GLMs, GBMs offer greater flexibility but less interpretability. Model validation uses holdout or cross-validation with actuarial metrics like Gini coefficient and lift charts. Exam MAS-II tests the gradient boosting algorithm, hyperparameter effects, and comparison with traditional GLM approaches.