Carry flow control and cancellation all the way to resource owners

Choose a buffer and record produced and drained bytes, not only token counts. Token strings have variable sizes, and streaming envelopes, metadata, and encoding add bytes beyond the text itself. A constant bytes-per-token value is useful only as an explicit approximation or bound for the chosen fixture. Separate application queues, socket buffers, proxies, and client-side buffering because a limit in one layer does not necessarily bound the others. The overall pipeline needs ownership and flow-control decisions at its actual bottlenecks.

In Python 3.13 streams, drain participates in transport write-buffer flow control, waiting when the high watermark is reached until the low watermark permits progress. That is a local transport contract, not a command to stop an independent inference engine. If generation runs in a separate task or process with an unbounded output queue, a blocked writer may simply move the backlog upstream. Bound that queue and choose whether to pause supported scheduling, reject additional work, or cancel according to the application contract.

Propagate cancellation using stable request identity to the engine's supported abort path, stop selecting new work, and wait for the relevant in-flight operations to reach a safe ownership boundary. An already launched kernel may finish before cleanup takes effect. Shared prefix blocks must remain allocated while other requests reference them. Releasing every referenced block immediately on socket closure can corrupt other work, while never releasing the canceled request's exclusive blocks wastes memory. Record each stage separately.

The pinned vLLM 0.10.2 AsyncLLM implementation routes cancellation of its generation coroutine through an abort call. That is useful implementation evidence but does not prove every proxy, client, wrapper, and transport path propagates disconnects identically. Test the deployed stack, including cancellation during prompt processing and decode, a slow reader, a disconnect after completion, and repeated cancellation. Cleanup should be idempotent at the application level so racing terminal events do not double-release or create duplicate completion notifications.

Measure generated-but-undelivered bytes, blocked writers, queued engine outputs, abort acknowledgments, remaining scheduled work, and reclaimable memory. After cleanup, submit a controlled probe that needs the released capacity and verify it can proceed without relying only on a counter decrement. Include partial failures and shared-cache references. A zero active-request count in the frontend can coexist with an engine still holding resources, so the acceptance oracle should cross the actual ownership boundary.

The queue lab models only a fluid production/drain imbalance. Reset, set both arrival λ=4 and service μ=2, predict backlog twelve after four seconds, then Play or take eight Steps. Reset and set both λ=2,μ=4 for a fresh comparison draining the initial four units in two seconds. Edits retain time and are not live rate switches. The lab has no network buffers, cancellation, shared KV blocks, or finite queue cap; its growing backlog is a reason to define real flow control, not a simulation of it.