Schedule requests through explicit iteration-boundary states

A request can be accepted into a waiting queue without owning all resources needed for execution. Admission and scheduling are therefore different transitions. During a chunked prompt, record how many prompt tokens have been computed and which KV state is valid. Ordinary next-token decode becomes eligible only after the required prompt context is available. A scheduler can combine different requests' eligible work, but it must preserve each request's dependency chain and the serving engine's supported iteration semantics.

Orca introduced iteration-level scheduling as a way to change batch membership between model iterations. That does not imply arbitrary interruption within a running kernel. The versioned vLLM 0.10.2 scheduler source is a concrete implementation with request progress, running and waiting sets, and resource decisions; its names are implementation details rather than a universal state schema. Derive an abstract lifecycle first, then map its transitions to the actual engine events and counters used in the deployment.

After an iteration, a request may produce an output token, finish by EOS or a limit, continue, become preempted, or be canceled according to the supported policy. A terminal request should no longer be selected for new ordinary decode work. Its exclusively owned memory can be released only when safe with respect to in-flight operations; shared cache blocks need correct reference handling. Completion delivery and resource release are related events, not necessarily the same instant.

Preemption introduces another distinction between logical output progress and currently resident KV state. A request can retain already delivered output while losing cache residency and later recomputing context. Recompute consumes work but must not duplicate committed output or terminal notifications. Use stable request identity, transition reasons, and versioned progress records to test races among completion, cancellation, and a new scheduling decision. An entry disappearing from one queue is not proof that every downstream worker stopped owning its resources.

Create a deterministic small fixture with different requested output lengths, one queued arrival, and one cancellation. Inspect the active set after each boundary, prompt progress, emitted token sequence, finish reason, and allocation ownership. Verify that a completed slot can be reused without delaying the completed response until the longest request ends. Also test a preempted request's reentry so logical continuation is distinguished from restarting the entire external request.

The Markov lab provides only a hypothetical pending-to-terminal probability process. Reset, set both p=0.5 for state 0→1 and q=0 for 1→0, predict pending probability 1/16 after four Steps, then inspect the exact readout. Reset and set both p=0.25,q=0 for a separate comparison. The terminal state is absorbing in this model. There are no arrivals, token dependencies, batch membership, cache blocks, or sampled request trajectories; the lab illustrates a declared state transition without claiming to simulate continuous batching.