Markov Chains: Transition Matrices and Stationary Distributions

Transition Matrices

A discrete-time Markov chain has the memoryless property: P(X_{n+1}=j | X_n=i, X_{n-1},...) = P(X_{n+1}=j | X_n=i) = p_{ij}. The transition matrix P has entries p_{ij} with rows summing to 1. The n-step transition matrix is P^n, giving probabilities of moving between states in n steps. States are classified as transient (eventually left forever) or recurrent (returned to infinitely often). Recurrent states may be positive recurrent (finite expected return time) or null recurrent. An absorbing state has p_{ii}=1.

Stationary Distributions

A stationary distribution pi satisfies pi*P = pi and sum of pi_i = 1. For an irreducible, positive recurrent chain, a unique stationary distribution exists and pi_j = 1/E[T_j] where T_j is the return time to state j. If the chain is also aperiodic, the long-run proportion of time in each state converges to the stationary distribution regardless of the starting state. In actuarial applications, Markov chains model bonus-malus systems, no-claims discount progression, and multi-state insurance transitions. Exam MAS-I tests chain classification, stationary distribution computation, and long-run probability calculations.