Repair must restore the intended committed generation

Read-triggered repair uses evidence found during access to address a missing or bad copy. Background recovery can restore missing redundancy even when no client requests the affected object. Ceph Squid documents read repairs as well as recovery and backfill states, but those mechanisms should not be projected onto a managed provider whose internals are not specified. The general distinction is useful: a successful response proves an operation succeeded under its contract, not that every desired replica or fragment has already been restored.

Cold data is a particular reason to retain background mechanisms. An access-only strategy might leave a missing fragment unrepaired indefinitely until a later failure removes another. On the other hand, aggressive background work competes with foreground reads and writes. Measure pending repair work, oldest degraded age, reconstruction failures, and restored placement, alongside client latency. A count of started repair tasks is not the same as the number of fragments written, verified, and incorporated into the correct current placement.

A checksum tells whether bytes match their associated reference, not whether they represent the newest committed generation. Two replicas can each have internally consistent but different historical values. The replication protocol's logs, versions, commit evidence, and peering rules determine what is authoritative under its fault model. Choosing the largest wall-clock modification time or taking an unqualified majority of hashes is not a universal replacement for that protocol. Preserve conflicting evidence before an operator action destroys the only newer copy.

Erasure repair also requires correctly indexed valid fragments from one encoded generation. Combining an old fragment with new fragments can produce an invalid or unintended reconstruction even if each fragment passes its own local check. A repair write must respect concurrent updates through the backend's generation and coordination rules. The examples use a hypothetical fixed stripe during repair to make arithmetic clear; they do not claim an S3 ETag is an internal replica version or that managed APIs expose the same evidence as a Ceph administrator.

Surviving disks, network links, decode CPU, target writes, and legal placement space can each limit restoration. More workers do not increase throughput once the actual bottleneck saturates, and they can increase foreground latency or failures. An aggregate bandwidth model is useful only if its counted bytes correspond to the stated resource. Distinguish source reads from network transfer and target writes instead of casually counting each as one equivalent byte on every architecture.

The queue lab illustrates a normalized repair-work backlog. Reset, set both λ=3 and μ=2, predict growth from four items, then Play or Step. Reset and set both λ=2,μ=3 for a separate drainage comparison. Parameter edits retain elapsed time and recompute from the initial four, so this is not a live rate switch. There are no version conflicts, disks, coding matrices, or finite buffers in the instrument. Use its fixed-rate imbalance to motivate measuring real completed work, not as a forecast of a cluster's recovery time.