Neural Network Basics for Actuarial Applications

Architecture

A feedforward neural network consists of an input layer (one node per predictor), one or more hidden layers, and an output layer. Each node in a hidden layer computes a weighted sum of its inputs plus a bias term, then applies a nonlinear activation function: h = g(w'x + b). Common activation functions include ReLU (max(0, z)), sigmoid (1/(1+e^(-z))), and tanh.

The output layer uses an activation appropriate for the task: linear for regression, sigmoid for binary classification, softmax for multi-class classification.

Training

Weights are learned by minimizing a loss function (e.g., mean squared error for regression, cross-entropy for classification) using backpropagation and gradient descent. The backpropagation algorithm efficiently computes gradients of the loss with respect to each weight using the chain rule. Stochastic gradient descent (SGD) and its variants (Adam, RMSprop) update weights using mini-batches of data for computational efficiency.

Hyperparameters include: the number and size of hidden layers, learning rate, batch size, number of epochs, and regularization strength. These are tuned using validation data or cross-validation.

Actuarial Considerations

Neural networks can model complex nonlinear relationships that GLMs and decision trees might miss. However, they require large datasets, are computationally expensive, and lack the interpretability of GLMs. In actuarial applications, neural networks are used for image-based claims processing, telematics data analysis, and as components of ensemble models. For Exam SRM, the focus is on understanding the architecture, the role of activation functions, how backpropagation works conceptually, and the tradeoff between model complexity and interpretability.