Bayesian Statistics Fundamentals for Exam SRM

Bayesian Framework

Bayesian statistics treats parameters as random variables with probability distributions. The prior distribution p(theta) represents beliefs about theta before observing data. The likelihood L(data | theta) gives the probability of the data given theta. Bayes' theorem combines these: the posterior distribution p(theta | data) is proportional to p(data | theta) * p(theta). The posterior is updated beliefs after observing data.

The key equation: posterior is proportional to likelihood times prior. The normalizing constant ensures the posterior integrates to 1 but is often unnecessary for identifying the posterior's distributional form.

Conjugate Priors

A prior is conjugate to a likelihood if the posterior belongs to the same distributional family as the prior. This greatly simplifies computation. Key conjugate pairs for Exam SRM: Beta prior with Binomial likelihood yields Beta posterior. Gamma prior with Poisson likelihood yields Gamma posterior. Normal prior with Normal likelihood yields Normal posterior.

For example, if theta ~ Beta(a, b) prior and X | theta ~ Binomial(n, theta), then theta | X ~ Beta(a + x, b + n - x). The posterior mean is (a + x) / (a + b + n), a weighted average of the prior mean a/(a+b) and the data proportion x/n.

Point Estimation and Credible Intervals

Bayesian point estimates include the posterior mean (minimizes squared error loss), posterior median (minimizes absolute error loss), and posterior mode (MAP estimate). A 95% credible interval [L, U] satisfies P(L < theta < U | data) = 0.95. Unlike frequentist confidence intervals, credible intervals directly give the probability that the parameter lies in the interval (given the prior and data). For Exam SRM, understand the conceptual difference between Bayesian and frequentist inference and be able to compute posterior distributions for conjugate models.