Blocking, readiness, and completion are different contracts

A blocking interface may keep the calling thread inside an operation while progress depends on another component. The thread can be asleep rather than consuming CPU, but it cannot simultaneously execute your next handler. A nonblocking interface instead reports that the operation cannot presently make progress. That response is ordinary control flow. It is neither an empty message nor proof that the peer has failed. The application must save its current offset and arrange another opportunity to try.

An asynchronous completion interface accepts an operation whose result arrives later. Submission and completion are separate observations: acceptance alone does not provide bytes or establish success. Meanwhile, a readiness interface reports a condition associated with a resource; the application still performs the operation. Both can support overlapping connections, and both require error handling. Ask which exact API produces the byte count before deciding which component owns the transition from waiting to usable data.

Linux epoll reports readiness rather than performing a read for the application. Use nonblocking descriptors so a stale observation cannot trap an event-loop thread inside a blocking call. Another consumer or a change in state can alter what an attempted operation observes. Inspect its actual return value, including positive progress, end of stream, interruption, or the would-block condition. Protocol framing remains a separate responsibility: readable bytes do not automatically constitute a complete application message.

Level-triggered and edge-triggered notification policies change how the application revisits resources. In an edge-triggered design, preserve knowledge that work remains until nonblocking I/O reaches its stopping condition. If a fairness budget interrupts draining, place the resource on an application-ready list rather than blindly waiting for another edge. This combines progress with scheduling discipline. The resource state must say both where byte processing stopped and whether the dispatcher still owes it attention.

A thousand outstanding requests do not create a thousand processors or unlimited downstream bandwidth. Overlap can hide independent waiting, while CPU work, connection limits, buffer memory, and dependency capacity still constrain throughput. Bound outstanding operations and record how many are waiting, runnable, and terminal. An increase in concurrency is useful only if completed useful work improves within latency and memory budgets. A submitted operation that will eventually be abandoned still consumes some resources now.

Use the queue lab as a capacity analogy, not an epoll or completion simulator. Press Reset, set arrival rate to 2 and service rate to 3, predict whether the initial four-job fluid backlog drains, then press Play. For an independent comparison, press Reset again, explicitly set arrival rate to 4 and service rate to 3, predict growth, then press Play. These are fresh runs from four jobs, not a live rate change applied to an old backlog. The model cannot distinguish readiness from completion or expose notification races. Build a separate paper trace with operation identifier, attempted byte count, returned byte count, and next state to test that distinction.