Exponential Family Distributions for GLM Applications

Exponential Family Form

A distribution belongs to the exponential family if its density can be written as f(y; theta, phi) = a(y, phi) * exp((y*theta minus b(theta))/phi), where theta is the natural (canonical) parameter, phi is the dispersion parameter, and b(theta) is the cumulant function. Key properties: E[Y] = b'(theta) and Var(Y) = phi * b''(theta) = phi * V(mu) where V(mu) is the variance function. Members include Normal (V(mu) = 1), Poisson (V(mu) = mu), Gamma (V(mu) = mu^2), Binomial (V(mu) = mu(1-mu)), and Inverse Gaussian (V(mu) = mu^3).

GLM Connection

The exponential family is the foundation for GLMs. Each member has a canonical link function: theta = g(mu), which simplifies the sufficient statistics and estimation. For Poisson, theta = ln(mu) so the canonical link is log. For Binomial, theta = ln(mu/(1-mu)) giving the logit link. The deviance for each family follows from the cumulant function. Understanding the exponential family unifies the theory of GLMs and reveals why certain link-distribution combinations have desirable statistical properties. Exam MAS-I tests identification of exponential family members, their canonical links, and variance functions.