Stochastic gradient methods

Let the history contain all previous iterates and sampled information. A standard stochastic-gradient model requires the next estimate to have conditional mean equal to the true gradient at the current iterate. This is stronger and more useful than merely saying that the error averages to zero over an unspecified experiment. The current iterate depends on earlier randomness, so the relationship between a new sample and that history is central to the analysis.

For a finite sum of losses, sampling an index uniformly and taking that term's gradient gives an unbiased estimate of the average gradient, provided sampling is appropriate for the conditioned history. Nonuniform sampling usually needs inverse-probability weighting. Sampling without replacement across an epoch can be beneficial, but its conditional behavior differs from independent sampling, so a proof assuming fresh independent indices should not be applied without checking the change in sampling mechanism.

Apply the smoothness upper bound to a random step and then take conditional expectations. The exact-gradient descent term survives, while a variance penalty appears with the square of the step size. For a fixed nonzero step and persistent noise, this penalty generally prevents arbitrary precision. The iteration may enter a neighborhood rather than settle at one exact point. Smaller steps reduce the neighborhood but also reduce the speed of meaningful movement.

Averaging b independent, identically distributed unbiased estimates divides their variance by b. Correlation can remove that benefit, and a larger batch costs more evaluations. In classical stochastic-approximation settings, decreasing schedules with divergent total step length and finite total squared step length help balance persistent exploration against accumulated noise. Those two sums alone are not a universal convergence theorem: regularity, stability or boundedness, and noise conditions still have to be supplied.

For the unit quadratic, writing a stochastic gradient as x plus zero-mean noise makes the distinction especially clear. The conditional mean contracts like ordinary gradient descent, but the next squared error includes an extra variance contribution. A mean trajectory near zero therefore does not imply that individual runs are near zero. Reporting only averaged parameter coordinates can hide substantial fluctuations and poor realized objective values.

The interactive lab contains no random sampling: it displays the exact deterministic recurrence. Use it to predict the conditional mean of one stochastic update at the same x and η, then calculate the extra variance below. It cannot demonstrate stochastic fluctuations, minibatch variance, or a stochastic convergence theorem. In actual experiments, record seeds, sampling rules, learning-rate schedules, and distributions across runs so the empirical claim matches the mathematical model.