advanced24 min studyLesson 328 in Engineering

Negative caching without inventing absence

Cache well-scoped absence separately from permission failures and transient errors, with bounded recovery delay.

On this page

What you’ll understand

  • Distinguish authoritative nonexistence from an unsuccessful lookup.
  • Bound the visibility delay introduced by a declared negative lifetime.
  • Explain how an outage can become a high-hit-rate error cache.

Repeated lookups for a nonexistent object can waste origin capacity, so storing a negative result can be useful. The difficult part is deciding what the negative result means. A timeout, an unauthorized lookup, a lagging replica, and authoritative absence are different states. Collapsing them into the same long-lived entry can prolong an outage, hide a newly created object, or leak information across callers. Define the negative type, scope, and expiry before counting the saved requests.

A failure to return data is not proof of nonexistence

An origin can explicitly report that a key does not exist under a known namespace and consistency contract. That is different from a network timeout or an internal error that left existence unknown. Permission-denied responses may be intentionally indistinguishable from absence to a user, but that does not justify reusing the result for another authorization scope. Preserve typed outcomes and the same tenant, representation, and permission boundaries used for positive entries.

A negative entry should carry enough metadata to distinguish why it was stored and which source/version supports it. A lagging replica can report absence after creation, and an earlier absence read can pause before publishing a fresh-TTL entry much later. In the bound below, L_source must cover the time from creation through the last accepted stale-absence publication, including replica lag, read/response delays, and fill publication. If any of those delays is unbounded, a publication-anchored TTL alone gives no finite global visibility bound. Strict read-after-create can instead require authoritative validation or a guard rejecting outdated publication.

Thidden≤Tnegative+LsourceT_{\rm hidden}\leq T_{\rm negative}+L_{\rm source}
T_negative is the hard lifetime from each accepted negative publication. L_source is a proven upper bound from creation through the last accepted stale-absence publication, including replica lag and all read, response, and fill delays; use zero if no such publication occurs after creation. Both terms are seconds under compatible clocks. Unbounded publication delay or unenforced expiry removes this global bound.↔ Scroll wide equations horizontally.

Protecting the origin can prolong an error

Caching a transient failure can reduce repeated expensive attempts during an outage, but it is an error-backoff policy rather than authoritative absence. Give it an explicit short duration and recovery behavior. If every retry refreshes that duration, the cached error can remain indefinitely while the origin has recovered. A high hit ratio during this state may mean that many users are efficiently receiving the wrong availability outcome. Separate negative hits and transient-error hits from positive useful hits.

A broad error key can amplify one failed request across unrelated resources or users. A cache-wide backend-unavailable marker may be intentional, but then its blast radius and reopening checks must be explicit. Request coalescing, bounded retries, and circuit-breaking can control origin pressure without pretending that an unknown object is absent. Test recovery as carefully as failure: create a previously missing key, restore the origin, and change authorization while negative entries are warm.

Qorigin≈Qrequest(1−hnegative)Q_{\rm origin}\approx Q_{\rm request}(1-h_{\rm negative})
This simplified lookup budget uses requests/s and a negative-hit fraction h_negative for a fixed declared workload. It excludes positive hits, retries, refreshes, and coalescing. Reducing this rate does not prove the negative outcomes are correct.↔ Scroll wide equations horizontally.

Inspect correlation rather than assuming independent misses

Measure the reason, source, age, and scope of negative reuse without logging sensitive keys. Compare origin QPS with user-visible absence and error rates. A typo-heavy public search may benefit from caching true misses, while identifiers that are frequently created need a different lifetime. Put explicit limits on untrusted unique negatives so an adversary or random identifier stream cannot consume the entire cache. Admission policy matters for negatives just as it does for positive objects.

The probability lab illustrates an assumption to question. Reset and set both p=0.2, n=10, predict two independent negative outcomes on average, then inspect. A real shared outage can make errors highly correlated, so this binomial distribution is not an outage forecast or a negative-cache implementation. Play highlights exact possible counts rather than sampling origin failures. Explain how typed outcomes and source scope would distinguish genuine absence from a correlated infrastructure failure before permitting either kind of reuse.

A new object behind a warm negative

At time 0 an authoritative read confirms absence and stores a negative entry with an absolute 30-second lifetime. The object is created at time 0.1. Assume no invalidation, no source lag, no later in-flight absence publications, and no sliding refresh.

  1. The negative remains eligible until time 30 under the stated absolute policy. A creation immediately after fill is almost the worst case for the remaining lifetime.

  2. The create can be hidden for 30−0.1=29.9 seconds. The cache did not lose the object; it reused a previously correct but now outdated absence observation.

  3. A strict read-after-create requirement needs invalidation, bypass with authoritative routing, or a version-aware mechanism. Merely observing many negative hits does not demonstrate that the visibility contract is satisfied.

The result

Negative lifetime is also a potential creation-visibility delay. The acceptable bound follows the product's semantics rather than the desired hit ratio.

Explore the model

Interactive lab

Interactive lab · probability

Independent negatives are not an outage model

Reset, set p=0.2, n=10;predict mean 2 independent negative outcomes. These binomial bars do not distinguish absence, authorization, and timeout and do not model correlated failures or negative-entry expiry.

Independent negatives are not an outage modelThese are exact binomial probabilities for independent trials with one fixed success probability. Playback tours the distribution; it does not resample outcomes. Current readings: Mean · np: 5; Variance · np(1−p): 2.5; Total probability: 1. All parameters are adjustable with the labeled controls below.03581000.070.150.220.3Successes kP(X = k)k = 0 · P = 9.77e-4
Exact probabilityHighlighted outcomeMean np
Mean · np
5
Variance · np(1−p)
2.5
Total probability
1
t = 0 s

These are exact binomial probabilities for independent trials with one fixed success probability. Playback tours the distribution; it does not resample outcomes. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forOutcome typing and correlation determine whether a saved origin request is a correct reuse.

Check your understanding

An origin lookup times out. Which interpretation is justified before applying an explicit error policy?

Try it yourself

Ten thousand repeated missing-key lookups/s have an 80% valid negative-hit ratio under a fixed workload. Estimate origin lookups/s using the simple budget. Then explain why the same hit ratio during an origin outage would need different interpretation.

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

Count only the fraction that reaches the origin, and keep correctness separate from reduced work.

Compare with the solution

The simplified rate is 10,000×0.2=2,000 origin lookups/s, excluding refreshes and retries. During an outage, an 80% transient-error hit rate may reduce load while prolonging user-visible failure or hiding recovery. It must not be reported as proof of successful data reuse; typed outcome and recovery metrics determine whether the policy is acceptable.

Take it with you

  • Authoritative absence, permission denial, and unknown error are different outcomes.
  • Negative TTLs spend creation-visibility and recovery-delay budgets.
  • High hit rates can amplify errors unless outcome scope and recovery are measured.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.