Confusing R-squared with adjusted R-squared

R-squared always increases when you add a variable, even if it is useless. Adjusted R-squared penalizes for additional variables. Using R-squared alone for model comparison leads to overfitting.

Interpreting coefficients without considering the link function

In a logistic regression, the coefficient represents the change in log-odds, not the change in probability. In a Poisson GLM with log link, the coefficient represents the change in log-count. Always interpret on the correct scale.

Ignoring multicollinearity in regression

When predictors are highly correlated, individual coefficient estimates become unstable even though the overall model fit is good. Look for high VIF values (>5 or >10) and consider removing or combining correlated variables.

Using in-sample error for model selection

In-sample metrics like training MSE always decrease with complexity. Use cross-validation error, AIC, or BIC to compare models. A model that fits the training data perfectly may perform poorly on new data.

Misidentifying time series models from ACF/PACF

An AR(p) process has PACF that cuts off after lag p and ACF that decays. An MA(q) process has ACF that cuts off after lag q and PACF that decays. Mixing these patterns up leads to fitting the wrong model.

Forgetting to check stationarity before fitting ARMA models

ARMA models require stationarity. If the data has a trend or changing variance, you must difference or transform it first. Fitting an ARMA model to non-stationary data produces meaningless results.

Confusing PCA loadings with PCA scores

Loadings are the weights that define each principal component as a linear combination of the original variables. Scores are the values of the principal components for each observation. Using one when the other is required changes your answer entirely.

Over-pruning or under-pruning decision trees

A tree that is too deep overfits. A tree that is too shallow underfits. The optimal depth is found by cross-validation, not by visual inspection of the tree. Always use the 1-SE rule when selecting the pruning parameter.

Misinterpreting p-values

A p-value is not the probability that the null hypothesis is true. It is the probability of observing data as extreme as the sample, assuming the null is true. Stating this backward is a common conceptual error on the exam.

Neglecting to standardize variables before PCA

If variables are on different scales, PCA on the covariance matrix is dominated by the variable with the largest variance. Use the correlation matrix (or standardize first) when variables have different units.

Common SRM Mistakes