A JVP propagates one input direction through the program
Track primal values and tangents, calculate a shaped Jacobian-vector product, and distinguish directional differentiation from finite differences and a full Jacobian.
On this page
The question for today
What you’ll understand
- Determine the shapes of a real-valued JVP and its seed.
- Propagate tangents through elementary operations using the chain rule.
- Choose forward mode based on required directions rather than output count alone.
Suppose a program maps two parameters to three measurements and you want the response to one particular parameter perturbation. A full matrix of every partial derivative can be unnecessary. Forward-mode automatic differentiation carries a tangent beside each ordinary value and propagates the chosen direction through the executed operations. The result is a Jacobian-vector product at the current input, not a new evaluation at a slightly displaced input.
Primal and tangent values have different roles
For a differentiable map f from R^n to R^m, the input x has n components and its tangent seed v has the same input shape. The output f(x) and tangent J_f(x)v have m components. The Jacobian need never be materialized to compute this product. At each primitive, a derivative rule maps the incoming tangents to outgoing tangents while using the required primal values. Tangent propagation is linear in v at fixed x, even when f is nonlinear in x. This distinction is central to composing correct derivative transformations.
For a product z=ab, the tangent is b times the tangent of a plus a times the tangent of b. For a composition, the local tangent rules compose in program order. These rules differentiate the represented computation within their valid real domains; they do not remove floating-point rounding or make an invalid logarithm differentiable. Separate automatic differentiation from symbolic expression simplification and finite-difference perturbation. AD uses local rules at the represented values, while numerical checks later compare that result against independent evidence.
One direction does not reveal the whole derivative matrix
A single JVP answers a directional question. Recovering every Jacobian column with forward mode requires basis seeds or an equivalent batched transformation, with cost depending on the program and vectorization strategy. One directional propagation commonly costs a small multiple of evaluating the program, but this is not a universal constant-time promise for every primitive, batch layout, or runtime. Forward mode is especially natural when only a few input directions are needed; the best choice depends on the derivative product actually requested and the implementation's memory and operation costs.
The seed is held fixed when defining the directional derivative at x. If a later higher-order calculation makes v depend on x, differentiating J_f(x)v(x) also differentiates that seed unless the program explicitly treats it as constant. Array shapes and broadcasting can conceal this difference: a scalar seed expanded across several inputs asks for a sum of directional contributions, not an identity matrix. JAX's official jax-v0.8.0 autodiff documentation describes JVPs as a first-class transformation; runtime details should be checked against that pinned version when translating the mathematics into code.
Inspect a local derivative before trusting an implementation
Write down one small primal trace, its tangent trace, and the resulting dimensions. Test a zero seed and a scaled seed to catch accidental nonlinear seed handling. Compare independent directional finite differences at smooth, valid points, keeping perturbation scale and rounding in mind. A correct local rule can still be composed incorrectly if a wrapper detaches a value, casts to an unsupported type, or changes an input shape. Inspect the actual transformed path and state the real-valued domain before interpreting a plausible-looking tangent as proof.
The function lab supplies an analytic scalar example, not an AD engine. Reset, set BOTH coefficient a=1.5 and position x=0.5, predict slope 2ax=1.5, then Play or Step. Reset and re-set BOTH a=1.5 and x=1; predict slope three. A tangent seed of −2 would give scalar JVPs −3 and −6 on paper. Edits retain elapsed time; playback traces the fixed quadratic and does not change the selected input or run tangent code. Use its displayed slope to inspect this one rule, not to claim the lab differentiates arbitrary programs.
Work it through
Differentiate a two-input, three-output map in one direction
Let f(x,y)=(x²,xy,y²), evaluate at (2,3), and use fixed input tangent v=(1,−1). All quantities are real and dimensionless.
The primal output is (4,6,9). The Jacobian rows are (2x,0), (y,x), and (0,2y), giving rows (4,0), (3,2), and (0,6) at this input.
Apply each row to v: 4×1+0×(−1)=4, 3×1+2×(−1)=1, and 0×1+6×(−1)=−6.
The JVP is (4,1,−6), with output shape three. It predicts the first-order change along this one direction; it is neither a scalar gradient nor the entire 3×2 Jacobian.
One input-shaped seed produced one output-shaped directional response without requiring a general full-Jacobian computation.
The living notebook
Explore the model
Interactive lab · function
A scalar tangent rule you can inspect
For EACH run Reset, set BOTH a and x, predict, then Play or Step. Compare (a,x)=(1.5,0.5), slope 1.5, with (1.5,1), slope three. Compute a fixed-seed product on paper. Edits retain t; playback traces y=ax² and does not execute automatic differentiation or arbitrary vector functions.
- y at chosen x
- 0.608
- Tangent slope · 2ax
- 1.35
- Signed area · ax³/3
- 0.182
This lab uses only y = ax². Playback traces the curve within the fixed integration interval from 0 to x; it does not estimate numerical error or change the chosen point. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.
Look forA JVP applies a local derivative to a fixed input direction; the quadratic instrument illustrates only one analytic scalar rule.
Pause & reason
Check your understanding
Your turn
Try it yourself
For f(x,y)=(x+y,x²y) at (1,2), compute the JVP for fixed seed (2,−1). State the Jacobian and product shapes.
Take a moment to make a prediction or work it out on paper. Open a hint when you need one.
Give me a hint
Use rows (1,1) and (2xy,x²), then multiply by the two-component seed.
Compare with the solution
The 2×2 Jacobian at (1,2) is [[1,1],[4,1]]. Multiplying by (2,−1) gives (1,7), an output-shaped two-vector. The result describes this direction at the fixed input; changing the seed or input asks another derivative question.
Keep the idea
Take it with you
- Forward mode propagates an input-shaped tangent to an output-shaped response.
- A derivative is linear in its seed at a fixed differentiable input.
- Directional products, full Jacobians, and finite-difference evaluations have different costs and meanings.
Read further
Original sources and deeper explanations to follow at your own pace.
Optional progress, saved in this browser when available. Revisit any lesson at any time.
Lesson not marked complete.