The executed branch has a derivative only under local path assumptions

Consider f(x)=x² for x≥0 and f(x)=−x for x<0. At positive x the local derivative is 2x; at negative x it is −1. At zero the left and right derivatives differ, so the full map has no ordinary derivative there. Executing the nonnegative branch can still produce a branch-rule sensitivity of zero under an implementation's convention. That returned number does not prove differentiability. A finite perturbation that crosses the boundary can observe a different path and should not be interpreted using only the original branch's local Taylor rule.

PyTorch 2.9 records the graph of executed operations and recreates it each iteration, allowing ordinary dynamic control flow within its supported operations. Compiled transformations can impose different restrictions on Python decisions or loop primitives, so inspect the pinned runtime's rules. The predicate is not generally differentiated as if a Boolean comparison were a smooth arithmetic function. At a locally constant predicate, differentiating the active branch can be correct; at a switching boundary or discontinuity, a separate mathematical or surrogate convention is necessary.

For a fixed number of differentiable iterations, chain the derivative of each update. The intermediate states and their tangent or cotangent dependencies form the actual computation. If an input-dependent stopping condition changes the iteration count, nearby inputs can follow a different trace. A runtime that differentiates the executed trace does not automatically differentiate the discrete change in iteration count. The derivative of a converged mathematical solution, when it exists, can require an implicit-differentiation argument rather than simply calling a fixed iteration's derivative the solution's sensitivity.

Stateful effects inside loops add the same replay and mutation obligations as elsewhere. A random draw, external lookup, or global update can change the computation during rematerialization. A mask applied after division by zero can leave invalid operations in the differentiated graph; hiding their forward outputs does not necessarily remove NaN gradients. Arrange valid-domain computation before the undefined operation, using supported selection and indexing semantics, rather than relying on multiplication by zero afterward. Test both branches and their boundary neighborhoods, not only one convenient successful path.

Check a branch-sensitive program at smooth interior points, then separately characterize boundaries and invalid inputs. A gradient check at a kink can disagree even if the runtime follows its declared convention correctly; conversely, one passing interior check says nothing about the other branch. Record which mathematical target a test validates. For iterative solvers, compare several tolerances and iteration counts against a trusted target while measuring both residual and derivative error. A small primal residual alone need not prove that an unrolled derivative approximates the desired solution derivative accurately.

The quadratic lab displays only the smooth x² branch family. Reset, explicitly set BOTH a=1 and x=1, predict slope two, then Play or Step. Reset and re-set BOTH a=1 and x=0; its analytic slope is zero. Edits retain t and the animation traces a fixed quadratic. The instrument has no conditional or stopping predicate, so its zero slope cannot certify differentiability of the piecewise f at zero. Compute the left-branch derivative −1 on paper and explain why the mismatch, rather than the smooth lab picture, determines the boundary conclusion.