advanced28 min studyLesson 46 in Mathematics

Karush–Kuhn–Tucker conditions

Read constrained optimality as feasibility, balanced forces, and complementary activity, with explicit qualification assumptions.

On this page

What you’ll understand

  • Write the four KKT conditions with consistent signs.
  • Determine active constraints and verify a numerical certificate.
  • Distinguish sufficient convex certificates from necessary conditions requiring qualifications.

At a wall, the direction of greatest unconstrained improvement may be unavailable. The gradient can therefore remain nonzero at the best feasible point. Karush–Kuhn–Tucker conditions express how objective and constraint gradients balance, while recording which constraints actually matter. Their usefulness depends on understanding the logical direction of the claim: in a convex problem they can certify an optimum, but in general they do not turn every stationary point into a solution.

Four conditions, four distinct jobs

Write inequalities as fi(x)≤0f_i(x)\leq0 and equalities as hj(x)=0h_j(x)=0. Primal feasibility says the candidate obeys these rules. Dual feasibility says inequality multipliers are nonnegative. Stationarity balances the objective gradient with a weighted combination of constraint gradients. Complementary slackness requires each inequality multiplier times its constraint value to vanish. Together these conditions connect a feasible decision to an admissible lower-bound construction.

An inequality is active when its value is exactly zero. Complementary slackness forces the multiplier of every strictly inactive inequality to be zero. The converse is false: an active inequality may have zero multiplier if it is locally irrelevant to the optimum's value or redundant with other constraints. Thus counting nonzero prices is not the same as listing every geometric boundary touching the answer. Equality multipliers remain unrestricted in sign.

∇f0(x∗)+∑iλi∇fi(x∗)+∑jνj∇hj(x∗)=0,fi(x∗)≤0,hj(x∗)=0,λi≥0,λifi(x∗)=0\nabla f_0(x_*)+\sum_i\lambda_i\nabla f_i(x_*)+\sum_j\nu_j\nabla h_j(x_*)=0,\quad f_i(x_*)\leq0,\quad h_j(x_*)=0,\quad\lambda_i\geq0,\quad\lambda_i f_i(x_*)=0
These are stationarity, primal feasibility, dual feasibility, and complementary slackness for differentiable data with inequalities written in the ≤0 convention.↔ Scroll wide equations horizontally.

When a certificate is sufficient

Suppose the objective and inequality functions are differentiable and convex, the equalities are affine, and the candidate lies in their common domain. Nonnegative multipliers make the Lagrangian convex in the primal variable. Stationarity then gives its global minimum. Feasibility and complementary slackness make that minimum equal the primal objective at the candidate, matching a dual lower bound. This proves global optimality of any KKT point in this setting.

A constraint qualification is not needed for that sufficiency argument once a valid KKT certificate has been supplied. It matters when asking whether an optimum must possess such multipliers. For convex problems, Slater's condition is a standard sufficient qualification. For general smooth constrained problems, conditions such as linear independence of active constraint gradients can provide necessity at a local optimum. Without qualification, an optimum can exist while no KKT multiplier vector exists.

f0(x∗)=L(x∗,λ∗,ν∗)=q(λ∗,ν∗)≤f0(x)for every feasible xf_0(x_*)=\mathcal L(x_*,\lambda_*,\nu_*)=q(\lambda_*,\nu_*)\leq f_0(x)\quad\text{for every feasible }x
For convex differentiable objectives and inequalities with affine equalities, stationarity minimizes the Lagrangian, and complementary slackness closes the primal-dual gap.↔ Scroll wide equations horizontally.

Diagnose failure rather than memorize a checklist

Consider minimizing x subject to x2≤0x^2\leq0. The only feasible point is zero, so it is automatically optimal. Yet stationarity would require 1+λ(2x)=01+\lambda(2x)=0 at zero, an impossibility. The constraint has zero gradient there and admits no strictly feasible point. This example shows why writing down KKT equations and finding no solution does not, by itself, prove that the original problem has no optimum.

Numerical solvers report residuals rather than exact symbolic equalities. Feasibility errors, stationarity errors, and complementarity errors need scaling-aware tolerances; a tiny product can hide a large multiplier paired with a slightly infeasible constraint. The lab has no constraints or multipliers. Use its gradient readout as the unconstrained special case, then compare that zero-gradient stopping rule with the nonzero constrained gradient in the worked example. It cannot test KKT conditions for a user-supplied problem.

Find the active wall and its price

Minimize f0(x)=(x−3)2f_0(x)=(x-3)^2 subject to 1−x≤01-x\leq0 and x−2≤0x-2\leq0. Verify all KKT conditions at x=2x=2.

  1. At x=2, the lower constraint is 1−2=−11-2=-1 and the upper is 2−2=02-2=0. Primal feasibility holds. Complementary slackness requires the lower multiplier λ1=0\lambda_1=0, while the upper multiplier may be nonzero.

  2. The objective derivative is 2(2−3)=−22(2-3)=-2. Stationarity reads −2−λ1+λ2=0-2-\lambda_1+\lambda_2=0, so λ2=2\lambda_2=2. Both inequality multipliers are nonnegative.

  3. Complementarity gives 0(−1)=00(-1)=0 and 2(0)=02(0)=0. The objective is convex and the inequalities are affine, so this feasible KKT point is globally optimal with value one. Strict convexity also makes its decision unique.

The result

The upper wall has multiplier 2 and the inactive lower wall has multiplier 0. The objective gradient −2 is balanced by the active constraint gradient.

Explore the model

Interactive lab

Interactive lab · optimization

The unconstrained special case

Set η=0.5 and a nonzero x₀, then Step toward zero on f(x)=x²/2. Predict how its gradient shrinks. This lab shows ordinary unconstrained stationarity only; it does not include walls, active sets, multipliers, or KKT residual calculations.

The unconstrained special caseGradient descent on f(x) = x²/2: xₙ = (1 − η)ⁿx₀. Playback advances two discrete iterations per second. Values outside |x| ≤ 4 are explicitly reported. Current readings: Position x0: 2; Objective x²/2: 2; Iteration behavior: Stable step. All parameters are adjustable with the labeled controls below.−4−202402.174.356.528.7Parameter xf(x) = x²/2n = 0
ObjectiveDiscrete iterate
Position x0
2
Objective x²/2
2
Iteration behavior
Stable step
n = 0 · t = 0 s

Gradient descent on f(x) = x²/2: xₙ = (1 − η)ⁿx₀. Playback advances two discrete iterations per second. Values outside |x| ≤ 4 are explicitly reported. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forA raw zero-gradient condition is appropriate for this unconstrained quadratic, while constrained optima may require nonzero balancing multipliers.

Check your understanding

What does complementary slackness imply when an inequality is strictly inactive at a KKT point?

Try it yourself

Minimize x2x^2 subject to x≥0x\geq0. At the optimum, is the constraint active, and what multiplier satisfies KKT when it is written as −x≤0-x\leq0?

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

The objective gradient at zero already vanishes. Insert x=0 into the stationarity equation 2x−λ=0.

Compare with the solution

The optimum is zero and the constraint is active because −x=0-x=0. Stationarity gives 0−λ=00-\lambda=0, so λ=0\lambda=0. Primal feasibility, dual feasibility, and complementarity all hold. This is a concrete active inequality with zero multiplier, disproving the claim that every boundary touching the answer must have a positive price.

Take it with you

  • KKT combines primal feasibility, dual feasibility, stationarity, and complementary slackness.
  • Convexity makes a valid KKT certificate sufficient; qualifications address its necessity.
  • An active constraint can have zero multiplier, while an inactive one must.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.