Validate the target law and the serving benefit separately

Enumerate a tiny vocabulary and calculate accepted overlap, rejection probability, and conditional residual exactly. Include p=q, disjoint supports, target-only tokens, proposal-only tokens, near equality, and rejection at every position of a short block. Confirm that the accepted-plus-corrected mixture equals the target token by token and that the zero-rejection branch never normalizes 0/0. Add greedy path and tie cases separately. For a block or tree, compare target conditionals after each committed prefix against independent ordinary model evaluation so a correct sampler is not fed corrupted logits or stale KV.

Pin target and draft weights, tokenizer, processors, stopping rules, dtype, kernels, cache layout, and generation mode. Batch shape and numerical precision can change probabilities even when the conceptual model is the same. Pinned vLLM 0.10.2 explicitly separates algorithmic tests from log-probability stability limits, while the original papers state their numerical setting. Define which mathematical law is being approximated and which tolerances the implementation supports. Quality similarity, a matching greedy corpus, or a collection of equal seeded strings can support particular observations but does not replace the stochastic branch argument.

Repeated independent draws from a fixed known prefix can test one conditional distribution. Whole generated sequences contain dependent token positions and many different histories, so pooling them as identical Bernoulli trials is generally inappropriate. Predefine outcomes, sample size, and testing or interval procedures, and account for multiple tested histories or categories. A failure to reject a difference does not prove equality; a test can lack power for small but important errors. Conversely, a rare discrepancy can occur under a correct sampler. Investigate branch and numerical evidence alongside the statistical result.

Use exact or appropriate proportion methods for small counts and boundaries, and preserve the actual experimental grouping when comparing prompts or hardware runs. An empirical acceptance average is also not a distributional correctness test: a biased heuristic can accept many tokens. Track correction and stop branches and validate them directly. If using a tolerance-based approximation, report that target and estimate its effect rather than call the procedure mathematically exact. Distinguish the algebraic proof under exact normalized probabilities from the finite-precision and pseudorandom implementation that the tests exercise.

Compare the same target decoding contract and a declared prompt, context-length, output-length, and arrival mix. Separate prefill, first-token latency, inter-token gaps, completion latency, throughput, peak memory, and acceptance-prefix statistics. Speculative output can arrive in bursts, so a mean cost per committed token does not describe every gap. Include warmup and compilation policy, request cancellation, errors, and dropped work. Avoid reporting only the easiest prompt slice or comparing a different sampling temperature or approximate verifier against an exact baseline without identifying the changed workload.

The sampling lab isolates one known-prefix event model. For each separate view, Reset, set both p=0.6 and n to 25 or 100, predict the sample-mean variance, then inspect. The exact variances are 0.0096 and 0.0024. Playback highlights probability outcomes and the normal comparison is approximate; no decoder is run and no random samples are collected. The display helps reason about finite evidence but cannot certify equality of a large vocabulary law, cache rollback, or serving latency. Those conclusions require the analytic, implementation, and matched-load checks above.