Residual Analysis and Model Diagnostics for Exam SRM

Residuals and Their Properties

The residual for observation i is e_i = Y_i - Y_hat_i. Residuals estimate the true errors epsilon_i but have different properties: they sum to zero, are correlated with each other (even when true errors are independent), and have unequal variances. Standardized residuals (dividing by their estimated standard deviation) and studentized residuals (using leave-one-out estimates) are preferred for diagnostics.

A studentized residual exceeding 2 or 3 in absolute value suggests a potential outlier. On Exam SRM, know how to identify outliers using residual plots.

Diagnostic Plots

Residuals vs. fitted values: look for non-linearity (curvature), non-constant variance (funnel shape), or other patterns. A good model shows random scatter around zero. Normal Q-Q plot: residuals should fall approximately along a straight line if normality holds. Deviations at the tails indicate heavy or light tails. Residuals vs. each predictor: check for nonlinearity that might require a transformation or additional terms.

Influential Observations

Leverage measures how far an observation's predictor values are from the center: h_ii = the i-th diagonal element of the hat matrix H = X(X'X)^(-1)X'. High leverage (h_ii > 2p/n) indicates an observation that could strongly influence the fit. Cook's distance D_i combines leverage and residual size to measure influence on all fitted values. D_i > 1 (or > 4/n) suggests a highly influential point. For Exam SRM, understand that an influential observation has both high leverage and a large residual; high leverage alone does not guarantee influence.