Numerical differentiation and competing errors

The forward difference subtracts f(x) from f(x+h) and divides by h. A Taylor expansion shows a leading error proportional to h when the second derivative is controlled. Sampling symmetrically on both sides cancels the even terms in the numerator, producing a central difference whose leading error is proportional to h squared. This cancellation is mathematical and useful, but evaluating the numerator still involves subtraction of nearby computed values. The truncation argument alone says nothing about their rounding errors.

The error order assumes enough smoothness near the evaluation point and a fixed function as h decreases. At a kink, a central stencil may converge to an average-like number even though the classical derivative does not exist. Near a domain boundary, a symmetric sample may be unavailable; one-sided higher-order formulas are possible but have different coefficients and uncertainty amplification. A stencil is not a universal replacement for differentiability. Verify the domain and the solution concept before interpreting the returned number.

Suppose each function evaluation has an absolute uncertainty bounded by ε. In a central difference, the worst numerator perturbation is at most 2ε, giving a derivative uncertainty at most ε divided by |h|. If a truncation bound behaves like C times h squared, the combined upper model has a bowl-shaped dependence on positive h. The best scale depends on the third derivative and on evaluation accuracy. Unit roundoff alone does not specify ε when the function evaluation itself has cancellation, iteration error, or noisy observations.

Differentiation of measured data is especially sensitive because refining the sample interval does not necessarily reduce measurement noise. Smoothing or fitting a local model can be sensible, but then bandwidth and model bias become part of the analysis. Report those choices. A graph that looks smooth after aggressive filtering does not establish that its derivative is correct. The desired observable may instead be an integrated quantity, which is often much less sensitive to high-frequency noise.

In a clean second-order regime, halving h should reduce the leading truncation error by roughly four. Comparing D at h and h/2 estimates the smaller-step error from their difference divided by three, provided the leading expansion dominates. A plateau or reversal suggests rounding, noise, nonsmoothness, or an implementation issue. Two agreeing approximations can still share a bias, so an independently solvable test function and a range of step sizes are more informative than one apparently stable decimal string.

Automatic differentiation propagates derivatives through the executed operations using the chain rule, avoiding a finite-difference step. It differentiates that program along its executed branches; it does not automatically validate a physical model or repair nondifferentiable logic. Complex-step differentiation can avoid subtractive cancellation for functions admitting a suitable analytic complex extension, but operations such as absolute value or branch-dependent clipping can invalidate the argument. Choose the technique according to the mathematical function and the implementation, rather than treating every derivative API as interchangeable.