Glauber dynamics and Gibbs sampling

Let the configuration space be a finite Cartesian product and let π be a strictly positive probability on every configuration. Choose a coordinate i with fixed positive probability rᵢ, summing to one, and redraw its value from π conditional on all other coordinates. Other coordinates remain unchanged. The conditional denominator is the sum of target weights over possible values at that site with the surrounding configuration held fixed. It is not the global normalization constant, which cancels. This cancellation is the practical attraction when π is known only through unnormalized positive weights.

If two distinct configurations differ only at coordinate i, their transition probability is rᵢ times the appropriate conditional mass. Multiplying by π of the source shows a symmetric product of their two target weights divided by the same conditional denominator. This proves detailed balance for that update and, by summing over coordinates, for the random-scan kernel. State-independent coordinate-selection probabilities matter to this elementary proof; arbitrary state-dependent scan choices can change the invariant law unless their effects are explicitly compensated.

With strict positivity on the full finite product, any configuration can be reached by changing coordinates one at a time, because every required conditional draw has positive probability. Redrawing the current value also has positive probability, giving a self-loop. Thus the sampler is irreducible and aperiodic, with unique stationary law π and convergence from every start. This conclusion does not provide a useful bound on the number of updates: conditional probabilities may be tiny, and competing high-probability regions can create small conductance.

If some configurations have zero target mass, restrict the state space to the positive support and check its single-site connectivity. Correct conditional updates can preserve π while leaving several closed components, producing multiple stationary laws. Hard constraints or zero-temperature limits often require this extra analysis. Systematic scans also preserve π because each coordinate-update kernel preserves it, but the composition is generally not reversible and its time unit is a whole scan rather than one random coordinate update. Infinite-volume Gibbs-state uniqueness is a further question and is not settled by uniqueness on every finite box.

For independent coordinates, a conditional redraw ignores the neighbors, yet a single-site update still refreshes only one coordinate. Unselected sites retain their initial values. Coupling copies with the same selected sites and the same conditional draws shows why a coupon-collection scale can arise for many independent coordinates. For interacting coordinates, identical site choices and random numbers need not immediately erase discrepancies because the conditional thresholds depend on neighboring spins. Contraction arguments must quantify how much disagreement one update can create or remove.

This also explains the difference between preserving the target and drawing an exact independent sample from it. Starting already from π, every subsequent marginal remains π; starting elsewhere, one conditional update usually leaves the joint law different. Successive stationary observations can still be correlated, affecting Monte Carlo error even after initialization bias is controlled. A justified sampling analysis therefore combines an invariance proof with a convergence estimate and a variance calculation for the actual observable rather than treating a stationary formula as a complete performance certificate.