Point Estimation: MLE, MOM, and Minimum Variance

Maximum Likelihood Estimation

MLE finds the parameter values that maximize the likelihood L(theta) = product of f(x_i; theta). In practice, we maximize the log-likelihood. MLEs are consistent, asymptotically normal, and asymptotically efficient (achieving the Cramer-Rao lower bound). For regular models, the MLE variance is approximately 1/I(theta) where I(theta) = -E[d^2/dtheta^2 ln f(X; theta)] is the Fisher information. The invariance property states that if theta-hat is the MLE of theta, then g(theta-hat) is the MLE of g(theta).

Method of Moments and MVUE

Method of moments (MOM) equates population moments to sample moments and solves for parameters. For a k-parameter distribution, equate k moments. MOM estimators are simple to compute but generally less efficient than MLEs. The minimum variance unbiased estimator (MVUE) has the smallest variance among all unbiased estimators. The Cramer-Rao lower bound provides a floor for the variance of unbiased estimators. The Rao-Blackwell theorem improves estimators by conditioning on sufficient statistics. Exam MAS-I tests derivation and comparison of these estimation methods.