Proxy retries: one user operation can create many attempts

One retry means at most two attempts: the original plus one repeat. If a client can make three attempts and each traversed proxy can make two upstream attempts, the downstream can receive as many as six attempts for one original operation. Add another independently retrying layer and the multiplication continues. This is a worst-case upper count assuming every layer reaches its local cap and every attempt reaches the next layer. Eligibility rules, rejections, cancellation, deadlines, and successes can reduce actual work.

Distinguish logical operations, downstream attempts, upstream attempts, and successful completions in telemetry. A proxy receiving one request may start several upstream attempts, while a retried user request can look like a new downstream request unless identifiers preserve the relationship. If the upstream committed an operation and its response was lost, a connection reset does not prove the operation failed. Retrying a write requires application semantics such as an idempotency key and a defined deduplication contract; transport-level retry machinery alone cannot promise exactly-once effects.

A per-attempt timeout bounds waiting for one try, while an overall deadline limits the logical request's total permitted duration across attempts and backoff. Envoy v1.40.0 documents retries within its overall request timeout, with route and header configuration affecting the policy. An application may have an even shorter deadline that needs propagation and cancellation. A retry started with too little remaining time can consume backend work that cannot produce a useful response. Hedging is different again: concurrent attempts can overlap rather than waiting for an earlier attempt to finish.

A retry budget limits how much extra retry work is allowed relative to an explicitly defined population, often active requests in a proxy's cluster rather than a global count of user operations. Envoy's active-retry resources and retry-budget mechanism have documented scopes; do not interpret them as an automatically coordinated fleet percentage. Backoff and jitter reduce synchronization but do not create capacity. Retry only eligible failures, choose whether a different host is useful, and avoid immediate retries of persistent authorization failures or deterministic validation errors.

During a controlled failure, compare original-operation rate with attempted backend rate and completed useful work. Record the reason for each retry, consumed deadline, retry overflow, cancellation, and duplicate-effect handling. A falling user-visible error rate may be bought with sharply increased backend CPU and worsen other users' latency. Test a full dependency outage and a slow response after commitment, not only a single quick reset. Coordinate policy changes across layers so disabling one proxy retry does not leave a larger hidden SDK amplification untouched.

The queue lab illustrates changed offered work, not retry execution. Reset, set λ=2 and μ=3, predict the initial four-unit backlog drains in four seconds, then Play or Step. Separately Reset, set λ=4 and μ=3, predict growth of one unit per second, then Play or Step. Treat each unit as a backend attempt only for this synthetic comparison. The instrument does not generate retries, preserve operation IDs, implement backoff, or cancel side effects. Explain how more attempts can cross a service-capacity boundary even when original user demand remains unchanged.