Transition kernels and communicating classes

For a state space S, the entry P(x,y) is the conditional probability of the next state y given the present state x. Every entry is nonnegative and every row sums to one. Time homogeneity means the same kernel is used at every step; it does not mean the chain starts in equilibrium. The Markov property says that conditioning on the entire past gives this same next-state law once the present state is known. It concerns conditional distributions rather than deterministic independence between consecutive states.

If the initial law is a row vector μ, its law after n steps is μPⁿ. Matrix multiplication sums over intermediate states, giving the Chapman–Kolmogorov identity. On a countable space these sums remain well defined because their terms are nonnegative. A positive entry of Pⁿ means at least one positive-probability route of length n exists. A zero in P itself therefore rules out a direct jump but need not rule out eventual access through other states. Keeping the direction of multiplication fixed prevents balance equations from being inadvertently transposed.

State x reaches y when some nonnegative power of the kernel has a positive x,y entry. Allowing power zero makes reachability reflexive. Two states communicate when each reaches the other, and this equivalence relation partitions the space into communicating classes. A class is closed when no transition leaves it. An irreducible chain has one communicating class, so every state can eventually reach every other. A closed class can still have many outgoing arrows internally; closedness refers only to crossing its boundary.

For a finite chain, every trajectory eventually enters some closed communicating class with probability one, and each such class is positive recurrent. States outside the closed classes are transient. Finiteness is doing real work: a deterministic chain moving from n to n+1 on the nonnegative integers has no closed communicating class of this kind and never settles into a finite region. A reducible finite chain has stationary laws supported on its closed classes, while the weights assigned to those classes depend on how a stationary law is chosen rather than on a unique global equilibrium.

For a state with at least one positive-probability return, its period is the greatest common divisor of all positive return lengths. Communicating states have the same period. In an irreducible chain, period one is called aperiodicity. A positive self-loop forces period one, but a self-loop is sufficient rather than necessary: return paths of lengths two and three also give greatest common divisor one. An isolated transient state with no possible return needs separate treatment instead of assigning a misleading period from an empty list.

Irreducibility answers whether movement between states is possible; aperiodicity answers whether an arithmetic clock obstructs convergence. A deterministic cycle is irreducible on its vertices but its point-mass initial law rotates forever. Replacing a kernel by the average of itself and the identity, called making it lazy, retains communication while supplying self-loops. This changes the actual transition clock and generally changes mixing times, so laziness should be stated as a modification of the chain rather than silently assumed in an argument about its spectrum.