Dimension Reduction Techniques Beyond PCA

Principal Component Analysis Review

PCA finds orthogonal linear combinations of variables that maximize variance. The first principal component is the direction of maximum variance, the second is orthogonal and captures the next most variance, and so on. Eigenvalues of the covariance (or correlation) matrix give the variance explained by each component. The scree plot and cumulative proportion of variance guide how many components to retain. PCA is unsupervised, meaning it does not use the response variable. It works best when variables are linearly related and the main structure is captured by variance.

Alternative Methods

Factor analysis differs from PCA by modeling observed variables as linear combinations of fewer latent factors plus unique error. Partial least squares (PLS) finds components that maximize covariance with the response, combining dimension reduction with prediction. Multidimensional scaling (MDS) preserves pairwise distances in reduced dimensions. For non-linear structures, kernel PCA applies PCA in a transformed feature space. In actuarial applications, dimension reduction handles high-dimensional rating variables, telematics data, and multivariate loss analysis. Exam MAS-I covers PCA mechanics, interpretation, and the conceptual basis for alternatives.