A VJP pulls an output sensitivity back to the inputs

For a real map f:R^n→R^m, a cotangent u has m components and the reverse product J_f(x)ᵀu has n. This is often called a vector-Jacobian product when written with a row seed uᵀJ; here column notation makes the transpose explicit. For a scalar loss, seed one yields its input gradient. For a vector output, choosing all ones gives the gradient of the sum of outputs, not every Jacobian row separately. The seed's meaning belongs in the API contract and the mathematical question.

The reverse seed is held fixed for this product. A loss L=uᵀf(x) has gradient Jᵀu when u is constant; if u depends on x, a total derivative includes the additional dependency unless it is intentionally detached. This lesson uses real arrays and Euclidean inner products. Complex differentiation requires conventions about conjugation and real objectives that cannot be inferred by swapping an ordinary transpose into these formulas. State dtype, shape, and seed structure before comparing two framework results that may implement different derivative conventions.

If an intermediate feeds two downstream operations, both paths contribute to its cotangent. Reverse accumulation sums them; overwriting one with the other silently loses a chain-rule term. For z=a+b, the output cotangent passes to both inputs. For z=ab, each input receives the output cotangent multiplied by the other input's saved primal. This local reverse rule explains why forward values or equivalent recomputation are needed later. A graph's fan-out is mathematical reuse even when the two uses share storage or appear in different lines of source code.

One reverse product can be efficient for a scalar objective with many inputs, commonly costing a small multiple of the underlying program under suitable primitive assumptions. It still needs storage or recomputation and is not a constant-memory guarantee. Getting a full m×n Jacobian generally needs multiple seeds or batched transformations; batching can change memory and kernel costs. PyTorch 2.9's autograd.grad accepts output-gradient seeds and returns the requested input products, while its accumulation-oriented backward interface has different destination behavior. Do not confuse where gradients are stored with which derivative product was computed.

On a small example, write the Jacobian dimensions and compute one JVP and VJP by hand. Their inner products should satisfy uᵀ(Jv)=vᵀ(Jᵀu) for compatible real vectors. This adjoint check can catch orientation and seed-shape mistakes, but mutually consistent wrong rules can pass, so it does not replace independent derivative checks. Test accumulation through repeated uses and verify that a zero output seed produces zero input sensitivity within the valid domain. Numerical tolerance should reflect magnitude and floating-point behavior rather than hiding shape errors.

The matrix lab shows one fixed linear family. Reset, set BOTH diagonal scale a=1.5 and shear b=0.5, inspect A=[[1.5,0.5],[0,1]], then Play or Step. On paper, v=(1,−1) gives Av=(1,−1), while u=(2,1) gives Aᵀu=(3,2); both inner products equal one. Edits retain t; playback interpolates geometry, while readouts describe the selected final A. The instrument does not compute VJPs, transpose an arbitrary program, or turn the intermediate animation into a changing derivative rule.