Inference cost: count successful logical tasks

A logical task is the user-level unit whose outcome matters. It can generate several model attempts, tool calls, fallback requests, and human reviews. Propagate a stable identity through that work and choose how delayed attempts are charged to a cohort. Otherwise the cost of an initial failed response can land in one window while its successful retry lands in another, making both ratios misleading. Count one final qualified success per eligible task unless the application explicitly defines another useful unit.

The acceptance oracle combines the required content quality with the relevant deadline and other constraints. A completed transport request is not automatically a useful result. Keep offered, admitted, attempted, completed, and accepted task counts separately, and report rejected or unresolvable cases. Fix the workload mix before comparing configurations; selectively serving easy tasks can reduce a ratio while abandoning the original service. When the oracle is estimated or involves human judgment, preserve its version, uncertainty, and evaluation procedure alongside the cost results.

Include failed generations, speculative branches not accepted, retried tool calls, evaluation, and relevant shared overhead under a documented boundary. If a human review step is included, use an explicit fictional allocation rule rather than imply that model tokens describe its entire cost. Cache hits, input processing, output generation, and external operations can carry different quantities and rate rules. Do not add an already totalled cost field to the line items that produced it, and do not count a retry twice merely because it appears in two telemetry systems.

An observed cohort can be summarized through its measured success fraction, but an expectation over future random cohorts is another calculation. Dividing an expected numerator by an expected success count does not generally equal the expectation of their ratio. Small cohorts can have zero successes, making an unconditional ratio undefined, and costs can correlate with difficult failures. Publish raw totals, workload classes, and uncertainty. A simulation or analytic model of future outcomes needs its own assumptions about those dependencies rather than inherit certainty from a clean-looking aggregate.

Replay or randomize a controlled workload with fixed task identity, oracle, deadline, and accounting horizon. Report the distribution across task classes rather than hide a poor difficult-task result inside an easy majority. A lower success cost can coexist with lower overall availability or much worse tail latency. State the required success rate and offered demand as constraints, and include all attempts caused by the policy. Compare complete service outcomes before using the ratio to decide which implementation should receive more traffic.

The probability lab illustrates a deliberately simple known-probability denominator. Reset, explicitly set p=0.8 and n=20, predict, then Play or Step. Compare a fresh Reset with p=0.9 and n=20 explicitly set; exact binomial mean successes are sixteen and eighteen under independent identical trials. Playback highlights that fixed distribution rather than drawing real outcomes. The lab has no retries, correlated tasks, cost numerator, quality oracle, or expected inverse-success calculation. It cannot promise that either real cohort achieves its mean or that a mean-ratio estimate is unbiased.