Principal Component Analysis: Dimensionality Reduction for Exam SRM

What Is PCA?

Principal Component Analysis (PCA) transforms p correlated variables into p uncorrelated principal components (PCs), ordered by the amount of variance they explain. The first PC captures the most variance, the second captures the most remaining variance (orthogonal to the first), and so on. PCA is an unsupervised technique used for dimensionality reduction, visualization, and preprocessing before regression.

Mathematically, the PCs are the eigenvectors of the covariance (or correlation) matrix, and the variance explained by each PC is its eigenvalue. The total variance is preserved: sum of eigenvalues = sum of original variances.

Implementation Details

Before applying PCA, standardize the variables (subtract mean, divide by standard deviation) if they are on different scales. Otherwise, variables with larger variance will dominate the first few PCs. Use the correlation matrix instead of the covariance matrix when variables have different units.

To choose the number of PCs to retain: the scree plot shows eigenvalues in decreasing order; look for an "elbow." Alternatively, retain enough PCs to explain a target percentage of variance (e.g., 80% or 90%). The Kaiser criterion retains PCs with eigenvalues greater than 1 (when using the correlation matrix).

Exam SRM Applications

PCA is used in actuarial modeling to handle highly correlated predictors (an alternative to removing predictors for multicollinearity). Principal component regression fits a linear model using PCs as predictors instead of the original variables. This can improve prediction when there are many correlated predictors. For Exam SRM, understand the interpretation of PCs, the variance explained, and the distinction between PCA on the covariance vs. correlation matrix.