Stage recovery: preserve the visible token prefix under explicit assumptions

Bind recovery to request and generation identity, model and adapter versions, exact prompt tokens, accepted generated-token prefix, sampling configuration and relevant RNG state, stop and grammar state, penalties, and compatible cache representation. The cache’s covered input positions can lag a sampled or emitted token because a newly sampled token may enter the next model step before its KV is materialized. Record the actual next-input position and cache frontier instead of assuming one output token always equals one already stored cache position.

A compatible decoder can reconstruct missing KV by processing the exact known token prefix under the intended model and position semantics, but that consumes time and does not automatically reproduce every future floating-point or stochastic choice. Preserving already emitted token identities is a narrower claim than bitwise reproduction of an uninterrupted future stream. If necessary state or committed output records are unavailable, return the defined failure or restart signal rather than silently present a new generation as continuity of the old one. External tool side effects require their own idempotency contract.

A server can send token five and fail before receiving its acknowledgment; the client may or may not already have displayed it. Resending by text alone can duplicate visible output, while skipping it can lose output for the client that never received it. One proposed solution uses immutable token records keyed by request, generation, and sequence, a durable replay source, and a client that deduplicates those identities over the claimed reconnect and crash scope. Server ownership must be fenced so an old decoder cannot append a conflicting record after recovery.

These assumptions need implementation evidence. An in-memory token buffer does not survive arbitrary process or node loss, and a transport acknowledgment need not mean the application durably recorded or displayed a token. Define client acknowledgment and retention, log completeness, conflict detection, and rollback behavior. A lost handoff acknowledgment should resolve the same authoritative generation and epoch, not create two emitting owners. Exactly-once visible output is therefore an end-to-end protocol claim with failure limits, not a property inferred from retrying a network write.

Test source failure during copying, destination failure after readiness, ownership-commit acknowledgment loss, decoder death after sampling but before log commitment, and failure after send but before client acknowledgment. Verify the surviving authority, cache and token frontiers, exact replay keys, and stale-owner rejection. If no output has been committed, a new internal attempt can still need the same public request identity and bounded retry budget. Once output is visible, it cannot be retracted by quietly changing the model, prompt, or generation identifier. Preserve the client’s observed prefix in the oracle.

The reliability lab provides only majority availability counts. Reset, explicitly set replicas five and unavailable one, predict four available and threshold three, then Play or Step. Compare a fresh Reset with replicas five and unavailable three explicitly set, giving two available. There are no KV shards, output logs, client acknowledgments, ownership fences, or reconnects in this model. A majority flag cannot establish the stated continuity promise or tell whether the exact required generation state survived.