The chain rule, gradients, and Jacobians

Suppose g maps an open subset of n-space to m-space and is differentiable at a. Suppose f is defined on a neighborhood of g(a) and is differentiable there. A small input displacement first becomes Dg(a)h, then becomes Df(g(a))Dg(a)h. The neglected terms remain smaller than the original displacement because the first derivative is a bounded linear map and differentiability also gives continuity. This reasoning proves the chain rule without assuming that all second derivatives exist.

With column vectors, the Jacobian of g has m rows and n columns; the Jacobian of f has as many columns as g has output coordinates. Their product therefore has the expected final-output by initial-input shape. The outer derivative must be evaluated at the intermediate value g(a), not at the original point a unless those happen to coincide. A dimension check detects many errors before any arithmetic begins. Reversing the product generally describes a different operation or produces no defined matrix multiplication.

For a differentiable scalar function f, its derivative is naturally a linear functional: it accepts a displacement and returns a first-order output change. In Euclidean coordinates, the gradient is the unique column vector whose dot product with that displacement equals the derivative. For a unit vector u, the directional derivative is therefore the dot product of the gradient and u. The Cauchy–Schwarz inequality shows that the largest directional derivative is the gradient norm, attained in the gradient direction when that norm is nonzero.

A zero gradient makes every first-order directional derivative zero, but does not determine whether a point is a minimum, maximum, or saddle. Also, changing coordinates can alter what counts as a unit displacement. The familiar gradient formula uses the Euclidean inner product in the chosen coordinates; on a space with another metric, the same derivative functional is represented by a different gradient vector. Keeping the functional and its vector representation conceptually separate becomes essential when working with constrained coordinates or curved spaces.

Forward differentiation propagates an input perturbation through each Jacobian. Backward sensitivity propagation starts with a scalar objective's output covector and applies transposed Jacobians in reverse stage order. Both compute the same chain rule. The backward calculation is especially economical when there are many input coordinates but only one scalar objective. It does not invert the forward map and does not require the Jacobians to be square or nonsingular. Multiplication by a transpose is an adjoint action, not generally an inverse action.

For a time-dependent path r(t), the chain rule gives the rate of f along the path as the gradient dotted with velocity. This separates spatial sensitivity from actual motion. A large gradient need not produce a large observed rate if the path follows a level set; then the velocity is perpendicular to the gradient. Conversely, a non-unit velocity scales the rate with speed. Before calling a computed quantity a directional derivative, check whether the direction was normalized or whether the intended object was a derivative with respect to time.