research34 min studyLesson 899 in Engineering

Stop on verified task conditions or explicit limits, not the appearance of a finished trace

Define completion evidence, measurable progress, retries, and deadline reserves so an agent can report incomplete work without fabricating success.

On this page

What you’ll understand

  • Separate accepted completion predicates from heuristic progress signals.
  • Compute deadline feasibility including verification and finalization.
  • Detect repeated states without assuming that more steps imply more progress.

An agent can generate another plan, reflection, or tool call long after useful progress stopped. It can also stop early with a polished answer while a required effect remains unverified. Termination therefore needs a task-specific contract: what evidence establishes success, which limits bound continued work, and what state must be reported if those limits arrive first. Progress signals help allocate attention, but they are not interchangeable with completion. Longer traces and confident final language do not establish that the user's objective was achieved.

Define success using the required external conditions

Translate the task into acceptance conditions at the appropriate artifact or external-state boundary. A requested report can require supported conclusions; a requested change can require a particular artifact revision, validation, and a confirmed effect. Preserve which conditions are verified, unresolved, or contradicted. A tool's success status may establish only one operation's contract, not the whole task. A completion statement should reflect those conditions rather than the model deciding that its current explanation sounds sufficiently finished.

Budget exhaustion, a deadline, an unavailable dependency, and successful completion are different stopping reasons. The system can stop safely with a useful partial result and an explicit unmet condition instead of claiming success. Preserve pending operation IDs and actual external state so another turn or process can resume without repeating unknown effects. A research benchmark may define success with an automated verifier or environment signal; that definition is part of the measured result and does not automatically transfer to an open-ended deployment objective.

complete⁡  ⟺  ⋀j∈Grequiredverified⁡(gj)\operatorname{complete}\iff\bigwedge_{j\in\mathcal G_{\rm required}}\operatorname{verified}(g_j)
This is a declared completion contract for the required conditions, not a claim that every real-world goal can be perfectly verified. Each verification must match its artifact/version and evidence scope. Unresolved or contradicted required conditions cannot be converted into success by a heuristic score or a budget limit.↔ Scroll wide equations horizontally.

Progress signals need an interpretation and a failure test

Count newly verified required conditions, reduced unresolved dependencies, or an explicit task-specific potential rather than generated words or attempted calls. A valid progress measure can decrease when a source changes and earlier evidence becomes obsolete, so do not assume monotonically increasing done counts in a changing world. Detect repeated normalized states using versions, unresolved hypotheses, and pending effects. Repeating the same failed query with no new information may indicate a loop, while a documented transient retry can be justified by a different failure model.

Set retry and search limits under actual costs, and reserve resources for verification, state checkpointing, cleanup, and the final report. Predicted durations are uncertain, so a margin is a policy parameter to test rather than a mathematical promise that an action will finish on time. If a required check no longer fits, report the limitation or choose a permitted smaller path. Do not spend the last available tokens on another candidate and then mark it verified because the independent checker could not run.

Taction+Tverify+Tfinalize+Tmargin≤Tdeadline−TnowT_{\rm action}+T_{\rm verify}+T_{\rm finalize}+T_{\rm margin}\le T_{\rm deadline}-T_{\rm now}
All terms are nonnegative time estimates in one unit, with an explicitly chosen margin and positive remaining interval. This is a planning admission rule, not a guarantee against runtime variation. Actual completion evidence and elapsed costs must still be tracked; finalization is not interchangeable with verification.↔ Scroll wide equations horizontally.

Test both premature completion and unproductive continuation

Construct tasks with a plausible wrong answer, a missing required check, an uncertain write acknowledgment, a repeated failed observation, and a deadline that expires before a valid final action. Evaluate unsupported completion claims, useful partial reporting, duplicate effects, and wasted work alongside success rate. A stopping policy can look efficient by giving up on hard cases, or look persistent by looping without evidence. Inspect the original task requirements and the remaining state to distinguish those behaviors rather than ranking policies only by trace length.

The optimization lab gives a deliberately explicit numerical progress signal. Reset, set both η=0.5 and initial x0=2, predict x1=1 and x2=0.5 after one and two Steps, then inspect. Its objective x²/2 falls from 2 to 0.5 to 0.125. Reset and set both η=0 and x0=2; repeated Steps keep x=2 and objective 2. Each Step advances one iteration in x_n=(1−η)^n x0. There is no agent, tool, task verifier, or external effect. Iteration count alone does not imply progress even in this simple model.

Verification and reporting belong inside the remaining time

A controller has 12 seconds before a deadline. It reserves 2 seconds for finalization and 1 second of margin. Candidate A needs 6 seconds of action plus 4 of verification. Candidate B needs 5 plus 3. Assume fixed serial estimates for this calculation and both are otherwise allowed by the task.

  1. The action-and-verification allowance is 12−2−1=9 seconds. Candidate A requires 10, so it fails this declared admission rule even though its action alone would fit.

  2. Candidate B requires 5+3=8 seconds. Including finalization and margin gives 8+2+1=11 seconds, leaving 1 additional second under the toy estimates.

  3. If B's verification later fails or the actual action overruns, the controller must preserve that outcome and report the unmet condition. Passing the initial timing arithmetic did not establish successful task completion or a bound on real execution time.

The result

The stopping policy separates permission to try, predicted resource fit, actual progress, and verified completion. A deadline does not turn an unchecked result into a correct one.

Explore the model

Interactive lab

Interactive lab · optimization

Iteration count and task progress are different quantities

Reset, set both η=0.5 and x0=2, predict x1=1 and x2=0.5 after one and two Steps, with objectives 0.5 and 0.125. Reset and re-enter both η=0 and x0=2 for unchanged x=2 and objective 2 despite more Steps. Each 0.5-second Step advances one iteration of x_n=(1−η)^n x0 on x²/2. The model contains no agent goal, effect, verification, or deadline.

Iteration count and task progress are different quantitiesGradient descent on f(x) = x²/2: xₙ = (1 − η)ⁿx₀. Playback advances two discrete iterations per second. Values outside |x| ≤ 4 are explicitly reported. Current readings: Position x0: 2; Objective x²/2: 2; Iteration behavior: Stable step. All parameters are adjustable with the labeled controls below.−4−202402.174.356.528.7Parameter xf(x) = x²/2n = 0
ObjectiveDiscrete iterate
Position x0
2
Objective x²/2
2
Iteration behavior
Stable step
n = 0 · t = 0 s

Gradient descent on f(x) = x²/2: xₙ = (1 − η)ⁿx₀. Playback advances two discrete iterations per second. Values outside |x| ≤ 4 are explicitly reported. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forChoose a task-relevant progress signal and retain an independent completion condition.

Check your understanding

The compute budget ends before a required external check runs. What can the agent validly report?

Try it yourself

Nine seconds remain. Finalization needs 1 second and the declared margin is 1. An action estimate is 4 seconds and required verification 3. Does it pass the rule? If verification is not performed after an overrun, can the estimate justify success?

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

Include all four terms and distinguish planning fit from actual evidence.

Compare with the solution

The total is 4+3+1+1=9 seconds, so it fits exactly under the stated estimates. An overrun can still prevent verification; the original estimate cannot establish success. Preserve the actual state and report the required unverified condition rather than treating the equality as a real-time guarantee.

Take it with you

  • Completion requires the task's verified conditions, not a fluent ending or a resource limit.
  • Measure meaningful progress and detect loops with versioned state and effect identity.
  • Reserve verification and finalization, then report actual stopping reasons accurately.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.