Storage fairness: count operations and bytes separately

An IOPS limit counts operations in the relevant accounting domain; a bytes-per-second limit counts data volume. For fixed-size operations, multiplying operation rate by size relates the two. Small requests often reach an operation ceiling before a byte ceiling, while large requests can reach the byte ceiling first. This is an accounting model, not a performance prediction: device latency, read/write asymmetry, queue depth, random versus sequential access, and competing work can keep achieved throughput below either configured ceiling.

Distinguish an application-level lookup or checkpoint call from block requests. A lookup can be satisfied by a page cache without device I/O, or trigger several metadata and data reads. Filesystem merging, splitting, compression, copy-up, and writeback can change counts and timing. A “thousand requests” application metric therefore cannot be compared directly to a thousand block operations without tracing the path. For AI workloads, report request-size distributions and durability requirements alongside totals; a buffered write accepted into memory has not necessarily reached the persistence boundary the application needs.

Linux io.max identifies limits for a particular block device and resource direction. Inspect the effective hierarchy and underlying storage layout: a filesystem can sit above multiple layers, and the bottleneck may be elsewhere. Buffered writeback attribution depends on filesystem and kernel support and on which cgroup owns the writeback work. Shared inode patterns can complicate that ownership. A leaf’s application byte counter alone does not establish that the intended cgroup receives every relevant device charge. Compare application, filesystem, and device evidence before drawing a tenant-level conclusion.

A ceiling is also different from a minimum service entitlement. Throttling one writer can reduce interference without ensuring that a reader meets a tail-latency objective under every queue state. Remote storage adds server-side quotas, authentication, replication, and network bottlenecks beyond the local block controller. Define whether the claim covers local block traffic, a storage service account, a volume, or an object namespace. Enforce object and file permissions separately; traffic shaping cannot prevent an authorized path with an incorrect tenant check from exposing another customer’s checkpoint.

Keep the victim’s query distribution and cache state controlled, then vary aggressor request size, read/write mix, queue depth, and persistence behavior. A large buffered write test can expose writeback bursts that a direct-read test misses. Include a quiet baseline, a saturated aggressor, and a recovery phase while recording errors and unfinished work. If the victim appears faster when requests time out, completed-request latency alone is a misleading oracle. Match the measurement to the promised useful service and identify which device or remote dependency actually saturated.

Use the queue lab only for average normalized pressure. Reset, explicitly set arrival four and service three, predict growth from four pending items, then Play or Step. For an independent comparison Reset and explicitly set arrival two and service three before predicting drainage. Both rate values must be restored after Reset; an edit at nonzero time recomputes from the initial backlog. There are no blocks, page cache, request sizes, durability barriers, or I/O schedulers here. The real storage experiment supplies those missing mechanisms.