Regression Analysis Review for Exam MAS-I

Simple and Multiple Linear Regression

Linear regression models the relationship Y = X*beta + epsilon, where epsilon ~ N(0, sigma^2*I). The OLS estimator beta-hat = (X'X)^{-1}X'Y minimizes the sum of squared residuals. Key properties: beta-hat is unbiased, and its covariance matrix is sigma^2*(X'X)^{-1}. The coefficient of determination R^2 measures the proportion of variance explained. Adjusted R^2 penalizes for additional predictors. F-tests assess overall model significance, and individual t-tests evaluate each coefficient. Multicollinearity inflates standard errors and is detected by variance inflation factors (VIF).

Diagnostics and Model Selection

Residual analysis checks model assumptions: plots of residuals vs. fitted values detect non-linearity and heteroscedasticity, QQ plots assess normality, and leverage/influence measures identify outlying observations. Cook's distance combines leverage and residual size. Model selection criteria include AIC = -2*ln(L) + 2k, BIC = -2*ln(L) + k*ln(n), and adjusted R^2. Stepwise procedures (forward, backward, bidirectional) automate variable selection. Exam MAS-I tests regression mechanics, interpretation of computer output, and diagnostic evaluation.