research28 min studyLesson 884 in Engineering

Tool idempotency: preserve the effect identity across retries

Separate transport attempts from logical operations, define payload-bound deduplication, and account for provider retention and unknown outcomes.

On this page

What you’ll understand

  • Choose stable logical effect keys independent of retry attempts.
  • Specify payload equality and concurrent duplicate behavior.
  • Evaluate deduplication only within the provider's actual retention and scope contract.

A timeout can hide a committed external operation. Retrying is safe only when the service and client agree on what counts as the same logical effect. An idempotency key names that effect; it is not merely a random header added to each attempt. Record its scope, payload binding, concurrent behavior, and retention before claiming retries cannot create duplicates.

One logical operation can have many attempts

Derive or allocate an effect identity from the intended business operation and store it durably before dispatch. Reuse it for attempts that mean the same operation, including recovery by another worker when that is the declared scope. A new user intent can require a new identity even if its arguments match an earlier request. Conversely, generating a fresh key for each retry defeats deduplication. Do not confuse a transport request ID, model tool-call ID, worker incarnation, and provider effect key unless the mapping is explicitly designed.

Bind the key to canonical arguments, target resource, tenant, and operation version as required by the provider protocol. Reusing one key for a changed payload should be rejected or handled under a documented rule rather than silently returning an unrelated old result. Concurrent first attempts need an atomic provider-side claim or equivalent mechanism; a client-side lookup followed by an unprotected write can race. The service that commits the effect must participate in the property, not only the runtime's local journal.

Effect⁡(k,a)(m)=Effect⁡(k,a)(1),m≥1\operatorname{Effect}(k,a)^{(m)}=\operatorname{Effect}(k,a)^{(1)},\qquad m\geq1
This desired repeated-effect property applies to the same logical key k and equivalent arguments a within the service's stated scope and retention. It describes resulting effect state, not identical network timing or success of every response, and requires actual provider enforcement.↔ Scroll wide equations horizontally.

Retention and result semantics bound the guarantee

A provider may retain keys for a limited time, cache error results, reject mismatched parameters, or treat validation failures differently from executed requests. Stripe's idempotent-request documentation is one concrete API contract; its behavior must not be generalized to every tool. A workflow's recovery horizon can outlast the provider's deduplication record. After that window, repeating the same bytes and key may no longer suppress a new effect. Keep a longer-lived business ledger, reconcile resource identity, or stop for an explicit recovery decision as the application requires.

Idempotency does not guarantee the first operation succeeds, nor does it automatically authorize a retry after permissions change. An unknown result still needs an outcome lookup or a retry justified by the retained contract. Deduplication can return a prior receipt, report in-progress status, or reject a concurrent attempt under specific semantics. Record which occurred. A local unique constraint can prevent duplicate intent records while a remote write still executes twice if its own commit is outside the protected boundary.

Tdedup≥Tretry+Trecovery+Tdelivery marginT_{\rm dedup}\geq T_{\rm retry}+T_{\rm recovery}+T_{\rm delivery\ margin}
This is a conservative design condition when these nonoverlapping windows are measured from the same initial-effect reference point. Real timelines may require a different bound. Matching it alone does not implement deduplication or address delayed requests outside the assumed horizon.↔ Scroll wide equations horizontally.

Test duplicate attempts at the ambiguous boundary

Inject a lost response after provider commit, concurrent duplicate dispatches, changed arguments under one key, and a retry near or beyond key expiry. Inspect the provider's authoritative effect count as well as runtime results. Replay should preserve the key and payload digest. Verify that a new legitimate operation is not suppressed merely because it resembles an earlier one. Include authorization and budget treatment of duplicate transport attempts; a deduplicated external effect may still consume network, tool, or provider resources.

The cache lab only illustrates finite retained identities. Reset, set C=2 and s=4, predict zero hits among sixteen initial requests, then Play or Step. Separately Reset, set C=4 and s=4, predict twelve, then Play or Step. It uses LRU capacity rather than time-based key retention and has no provider commits. Use the lost working-set memory as a limited analogy, then test the actual retention clock and authoritative duplicate-effect behavior instead of equating cache hits with exactly-once execution.

Count attempts without counting duplicate effects

Ten distinct logical operations each commit once. Four extra attempts repeat existing keys and identical payloads while a fictional provider's deduplication records remain valid. The provider atomically enforces its stated repeated-effect property.

  1. There are fourteen transport attempts for ten logical operations, giving 14/10=1.4 attempts per operation.

  2. Four attempts are duplicates under the stated key and payload contract. The authoritative effect count remains ten, not fourteen.

  3. If the provider retains a key for forty-eight hours, a retry at hour thirty-six is within that stated window while hour sixty is twelve hours beyond it. The latter has no deduplication assurance from this retention rule alone.

The result

The ten-effect result depends on actual key enforcement and valid retention. Durable replay or local attempt counting alone cannot establish it.

Explore the model

Interactive lab

Interactive lab · cache

Retained identities are a bounded resource

Reset, set C=2 and s=4, predict zero initial hits in 16 requests, then Play or Step. Separately Reset, set C=4 and s=4, predict 12 initial hits, then Play or Step. This LRU address stream has no time-based retention, payload binding, provider result, or external effect.

Retained identities are a bounded resourceA fully associative LRU cache starts empty and requests address (i·s) mod 16. Hits move an address to the newest position; misses evict the oldest when full. Playback starts with 16 requests and adds two per second, capped at 4096 explicitly simulated requests. This toy stream is not a processor-cache benchmark. Current readings: Hits / requests: 0 / 16; Hit rate: 0%; Address-stream period: 16 addresses. All parameters are adjustable with the labeled controls below.Fully associative LRU cache after 16 deterministic requests: 0 hits and 16 misses.Most recent 16 requestsaddress = i × 3 mod 160M3M6M9M12M15M2M5M8M11M14M1M4M7M10M13MCurrent cache order · oldest → newest471013Latest address 13 · miss: inserted as newestToy address stream · period 16 · starts empty
Cached addressMiss / evictionMost recent request
Hits / requests
0 / 16
Hit rate
0%
Address-stream period
16 addresses
t = 0 s

A fully associative LRU cache starts empty and requests address (i·s) mod 16. Hits move an address to the newest position; misses evict the oldest when full. Playback starts with 16 requests and adds two per second, capped at 4096 explicitly simulated requests. This toy stream is not a processor-cache benchmark. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forTest the provider's actual key scope and retention instead of treating a local remembered ID as an external guarantee.

Check your understanding

Why does generating a new idempotency key for each retry undermine duplicate suppression?

Try it yourself

A proposed retry budget is six hours, offline recovery can add eighteen, and an additional nonoverlapping delivery margin is two hours. Find the stated conservative retention requirement. Is a twenty-four-hour record sufficient under that model?

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

Add the three windows measured from the common initial reference.

Compare with the solution

The bound is 6+18+2=26 hours. A twenty-four-hour record is two hours short under these assumptions. This arithmetic is a design check, not a claim about any provider's retention or a substitute for reconciliation after the bound is exceeded.

Take it with you

  • Logical effect keys must survive retries and bind the intended payload.
  • Provider enforcement and retention define the deduplication scope.
  • Replay and local uniqueness do not make remote effects exactly once.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.