Maximum Likelihood Estimator (MLE)The parameter value that maximizes the likelihood function L(theta) = product of f(x_i | theta). Found by setting the derivative of the log-likelihood equal to zero and solving for theta.Sufficient StatisticA statistic T(X) is sufficient for theta if the conditional distribution of the data given T does not depend on theta. By the factorization theorem, T is sufficient if L(theta) = g(T, theta) * h(X).Fisher InformationI(theta) = E[(-d^2/d(theta)^2) ln f(X|theta)]. Measures the amount of information a sample carries about the parameter. The variance of the MLE is approximately 1/I(theta) for large samples.Likelihood Ratio TestA hypothesis test that compares the maximized likelihood under the null to the maximized likelihood under the alternative. The test statistic -2*ln(L_0/L_1) follows a chi-squared distribution asymptotically.Conjugate PriorA prior distribution that, when combined with a specific likelihood, produces a posterior in the same family. Examples: Beta prior with Binomial likelihood, Gamma prior with Poisson likelihood.Posterior DistributionThe distribution of the parameter given the data: p(theta|x) is proportional to L(x|theta) * pi(theta). It combines information from the data (likelihood) and prior beliefs (prior distribution).Autoregressive Process AR(p)A time series model where the current value depends on p past values plus white noise: X_t = phi_1*X_{t-1} + ... + phi_p*X_{t-p} + epsilon_t. Stationary when the characteristic roots lie outside the unit circle.Moving Average Process MA(q)A time series model where the current value depends on q past error terms: X_t = epsilon_t + theta_1*epsilon_{t-1} + ... + theta_q*epsilon_{t-q}. Always stationary.Inverse Transform MethodA simulation technique that generates a random variable X with CDF F by setting X = F^(-1)(U), where U is Uniform(0,1). Works for any distribution with a computable quantile function.Acceptance-Rejection MethodA simulation technique that generates a random variable by sampling from a proposal distribution and accepting or rejecting based on a ratio. Useful when the inverse CDF is not available.Power of a TestThe probability of correctly rejecting a false null hypothesis. Power = 1 - beta, where beta is the Type II error probability. Higher power means the test is better at detecting true departures from the null.Confidence IntervalAn interval [L, U] constructed from sample data such that, in repeated sampling, a specified proportion (e.g., 95%) of such intervals contain the true parameter value.