advanced25 min studyLesson 501 in Engineering

A JVP propagates one input direction through the program

Track primal values and tangents, calculate a shaped Jacobian-vector product, and distinguish directional differentiation from finite differences and a full Jacobian.

On this page

What you’ll understand

  • Determine the shapes of a real-valued JVP and its seed.
  • Propagate tangents through elementary operations using the chain rule.
  • Choose forward mode based on required directions rather than output count alone.

Suppose a program maps two parameters to three measurements and you want the response to one particular parameter perturbation. A full matrix of every partial derivative can be unnecessary. Forward-mode automatic differentiation carries a tangent beside each ordinary value and propagates the chosen direction through the executed operations. The result is a Jacobian-vector product at the current input, not a new evaluation at a slightly displaced input.

Primal and tangent values have different roles

For a differentiable map f from R^n to R^m, the input x has n components and its tangent seed v has the same input shape. The output f(x) and tangent J_f(x)v have m components. The Jacobian need never be materialized to compute this product. At each primitive, a derivative rule maps the incoming tangents to outgoing tangents while using the required primal values. Tangent propagation is linear in v at fixed x, even when f is nonlinear in x. This distinction is central to composing correct derivative transformations.

For a product z=ab, the tangent is b times the tangent of a plus a times the tangent of b. For a composition, the local tangent rules compose in program order. These rules differentiate the represented computation within their valid real domains; they do not remove floating-point rounding or make an invalid logarithm differentiable. Separate automatic differentiation from symbolic expression simplification and finite-difference perturbation. AD uses local rules at the represented values, while numerical checks later compare that result against independent evidence.

f:Rn→Rm,JVP⁡f(x;v)=Jf(x)v∈Rmf:\mathbb R^n\to\mathbb R^m,\qquad \operatorname{JVP}_f(x;v)=J_f(x)v\in\mathbb R^m
x and v have n real components; the product has m. This lesson uses dimensionless inputs and outputs. With physical units, each Jacobian entry has output-component units divided by input-component units, and the tangent seed must have compatible units.↔ Scroll wide equations horizontally.

One direction does not reveal the whole derivative matrix

A single JVP answers a directional question. Recovering every Jacobian column with forward mode requires basis seeds or an equivalent batched transformation, with cost depending on the program and vectorization strategy. One directional propagation commonly costs a small multiple of evaluating the program, but this is not a universal constant-time promise for every primitive, batch layout, or runtime. Forward mode is especially natural when only a few input directions are needed; the best choice depends on the derivative product actually requested and the implementation's memory and operation costs.

The seed is held fixed when defining the directional derivative at x. If a later higher-order calculation makes v depend on x, differentiating J_f(x)v(x) also differentiates that seed unless the program explicitly treats it as constant. Array shapes and broadcasting can conceal this difference: a scalar seed expanded across several inputs asks for a sum of directional contributions, not an identity matrix. JAX's official jax-v0.8.0 autodiff documentation describes JVPs as a first-class transformation; runtime details should be checked against that pinned version when translating the mathematics into code.

z˙=ba˙+ab˙,Jf(x)(αv+βw)=αJf(x)v+βJf(x)w\dot z=b\dot a+a\dot b,\qquad J_f(x)(\alpha v+\beta w)=\alpha J_f(x)v+\beta J_f(x)w
The product rule uses real scalar primals a,b and their tangents. The second identity assumes f is differentiable at fixed x; α and β are real coefficients and v,w share the input shape. It does not assert linearity of f itself.↔ Scroll wide equations horizontally.

Inspect a local derivative before trusting an implementation

Write down one small primal trace, its tangent trace, and the resulting dimensions. Test a zero seed and a scaled seed to catch accidental nonlinear seed handling. Compare independent directional finite differences at smooth, valid points, keeping perturbation scale and rounding in mind. A correct local rule can still be composed incorrectly if a wrapper detaches a value, casts to an unsupported type, or changes an input shape. Inspect the actual transformed path and state the real-valued domain before interpreting a plausible-looking tangent as proof.

The function lab supplies an analytic scalar example, not an AD engine. Reset, set BOTH coefficient a=1.5 and position x=0.5, predict slope 2ax=1.5, then Play or Step. Reset and re-set BOTH a=1.5 and x=1; predict slope three. A tangent seed of −2 would give scalar JVPs −3 and −6 on paper. Edits retain elapsed time; playback traces the fixed quadratic and does not change the selected input or run tangent code. Use its displayed slope to inspect this one rule, not to claim the lab differentiates arbitrary programs.

Differentiate a two-input, three-output map in one direction

Let f(x,y)=(x²,xy,y²), evaluate at (2,3), and use fixed input tangent v=(1,−1). All quantities are real and dimensionless.

  1. The primal output is (4,6,9). The Jacobian rows are (2x,0), (y,x), and (0,2y), giving rows (4,0), (3,2), and (0,6) at this input.

  2. Apply each row to v: 4×1+0×(−1)=4, 3×1+2×(−1)=1, and 0×1+6×(−1)=−6.

  3. The JVP is (4,1,−6), with output shape three. It predicts the first-order change along this one direction; it is neither a scalar gradient nor the entire 3×2 Jacobian.

The result

One input-shaped seed produced one output-shaped directional response without requiring a general full-Jacobian computation.

Explore the model

Interactive lab

Interactive lab · function

A scalar tangent rule you can inspect

For EACH run Reset, set BOTH a and x, predict, then Play or Step. Compare (a,x)=(1.5,0.5), slope 1.5, with (1.5,1), slope three. Compute a fixed-seed product on paper. Edits retain t; playback traces y=ax² and does not execute automatic differentiation or arbitrary vector functions.

A scalar tangent rule you can inspectThis lab uses only y = ax². Playback traces the curve within the fixed integration interval from 0 to x; it does not estimate numerical error or change the chosen point. Current readings: y at chosen x: 0.608; Tangent slope · 2ax: 1.35; Signed area · ax³/3: 0.182. All parameters are adjustable with the labeled controls below.−2.4−1.201.22.4−9−4.504.59xy(x, ax²)
y = ax²TangentSigned integral
y at chosen x
0.608
Tangent slope · 2ax
1.35
Signed area · ax³/3
0.182
t = 0 s

This lab uses only y = ax². Playback traces the curve within the fixed integration interval from 0 to x; it does not estimate numerical error or change the chosen point. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forA JVP applies a local derivative to a fixed input direction; the quadratic instrument illustrates only one analytic scalar rule.

Check your understanding

For f:R²→R³ at a fixed differentiable input, what are the shapes of a JVP seed and its result?

Try it yourself

For f(x,y)=(x+y,x²y) at (1,2), compute the JVP for fixed seed (2,−1). State the Jacobian and product shapes.

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

Use rows (1,1) and (2xy,x²), then multiply by the two-component seed.

Compare with the solution

The 2×2 Jacobian at (1,2) is [[1,1],[4,1]]. Multiplying by (2,−1) gives (1,7), an output-shaped two-vector. The result describes this direction at the fixed input; changing the seed or input asks another derivative question.

Take it with you

  • Forward mode propagates an input-shaped tangent to an output-shaped response.
  • A derivative is linear in its seed at a fixed differentiable input.
  • Directional products, full Jacobians, and finite-difference evaluations have different costs and meanings.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.