Recurrence, transience and hitting times

The hitting time of A allows time zero, while the return time to x starts at time one. Thus a chain started in x has hitting time of x equal to zero regardless of its later behavior. State x is recurrent when its strictly positive return time is finite almost surely, and transient otherwise. A recurrent state is positive recurrent when that return time also has finite expectation; otherwise it is null recurrent. Recurrence type is shared within an irreducible communicating class, but a reducible chain can contain several different types.

The expected total number of visits to x, including the initial visit when starting there, is the Green sum of the diagonal transition probabilities. If the probability of ever returning is f less than one, successive returns and the strong Markov property give a geometric visit count with expectation one divided by one minus f. If f equals one, the sum diverges. Consequently recurrence is equivalent to divergence of that Green sum. This criterion tests return frequency over the entire future; observing no return during a finite simulation cannot establish transience.

For a target set A, the hitting probability h(x) equals one on A and is harmonic outside A: average h over the next-state distribution. The mean hitting time g is zero on A and satisfies a similar equation with an additional one for the transition just taken. These statements hold in the extended nonnegative sense when expectations can be infinite. In a finite chain that reaches A almost surely, the killed transition matrix on the complement has spectral radius less than one, and the associated finite linear system has a unique finite solution.

On a countable space, the first-step equations alone need not force uniqueness. The actual hitting probability is the smallest nonnegative solution with the stated boundary values, obtained by increasing finite-horizon hitting probabilities. The expected hitting time is likewise the minimal nonnegative solution of its duration equation. Arbitrary harmonic additions can produce algebraic solutions with no probabilistic interpretation. Before using an optional stopping identity, one must also justify its stopping and integrability assumptions; the existence of a formally constant martingale expectation at deterministic times is not enough.

For simple symmetric random walk on the integers, the origin is recurrent but its mean positive return time is infinite. A finite interval provides a useful approximation: stop on reaching either endpoint and solve there, then enlarge the interval. A fair walk started at i in the interval from zero to N hits N before zero with probability i/N, while its expected time to either endpoint is i(N−i). This expected duration grows with the interval, so a collection of finite absorption calculations does not automatically supply a uniform mean bound.

For a biased walk with probability p strictly between one half and one of moving right and q=1−p of moving left, a start one unit to the right of zero hits zero with probability q/p. From one unit to the left it hits zero almost surely. Combining these two possibilities after the first transition shows that the return probability from zero is less than one. This example separates a directional escape mechanism from periodicity: both biased and unbiased nearest-neighbor walks have period two, but only the biased walk is transient.