Admission is not eviction
Separate deciding what enters a cache from deciding what leaves under memory pressure.
On this page
The question for today
What you’ll understand
- Distinguish admission, replacement, expiration, and memory allocation.
- Explain how one-time scans can displace valuable resident data.
- Compare policies using request, byte, and origin-cost objectives.
A miss does not have to become a stored entry. Admission decides whether a candidate deserves cache space; eviction decides what existing data gives up that space. Keeping these decisions separate exposes why a cache can perform badly despite a plausible replacement rule. A stream of objects used once can push out a repeatedly useful working set unless the policy considers the value of admitting them.
Four decisions hide behind a set operation
An application may consider a response cacheable, an admission policy may reject it, a replacement policy may choose a victim, and an allocator may still lack room for its size class. Expiration separately determines whether an entry remains eligible for reuse by age. None of these is equivalent to the others. A miss counter cannot identify which decision caused absence, and an eviction count does not tell whether rejecting the incoming object would have been better.
Pure LRU favors recent access, while frequency-based admission can compare a candidate's estimated recent popularity with that of a proposed victim. Estimates can be approximate and aged so old popularity does not dominate forever. A policy tuned for stationary traffic can struggle after a workload shift. Avoid claiming one algorithm universally wins: the arrival distribution, object sizes, concurrency costs, and value of avoiding each origin fetch determine the useful comparison.
A scan can consume the whole budget
Suppose a cache holds a hot set that fits exactly, then a one-time scan accesses many new objects. An admit-everything LRU policy can evict the entire hot set while recording no future benefit from the scan entries. Rejecting some one-hit candidates can protect reuse, but a genuinely new popular object must eventually gain admission. Windowed or adaptive designs balance those pressures. Replay a trace with phase changes rather than optimizing only the final steady segment.
Object sizes matter as well as counts. One large object can displace several smaller valuable ones, and a high request hit rate can coexist with poor byte savings or expensive remaining misses. Metadata, allocator overhead, and policy bookkeeping consume memory and CPU. Compare actual resident bytes and origin work at the same total resource budget. A policy with a slightly higher simulated hit ratio may lose in production if its coordination cost overwhelms the saved fetches.
Use the lab as a baseline, not a policy tournament
Separate trace replay from live outcome measurement. A replay can estimate a defined policy's decisions for a recorded stream, but the trace may change when latency or admission changes. Warmup, expiration, invalidations, and concurrent requests need explicit treatment. In a live comparison retain correctness and freshness checks, observe origin cost, and sample policy CPU overhead. Do not compare a count-bounded simulation with a byte-bounded production cache as if their capacities were identical.
The cache lab implements only fully associative LRU and admits every miss. Reset and set both C=4, s=4, predict twelve hits in sixteen initial requests, then Step. Reset and re-set C=3, s=4; predict zero hits because the four-address cycle exceeds three entries. Its address stream is (i×s) mod 16 and capped at 4096 requests. There is no admission switch, variable size, TTL, or TinyLFU estimator. Explain how rejecting candidates would require a different trace and policy state rather than pretending a slider selects a new algorithm.
Work it through
A four-object scan displaces a hot set
A four-entry LRU cache initially holds equally sized hot objects A, B, C, D. A one-time scan requests E, F, G, H, each missing and admitted, then requests A. Assume no expiration or concurrent accesses.
Each new scan object requires one victim. After E, F, G, H are admitted, all four original hot objects have been displaced regardless of their initial recency order.
The next access to A misses, making five misses over these five requests. The four scan fills produced no reuse within this declared interval.
A hypothetical admission rule rejecting E through H would preserve A, B, C, D and make the final A access hit. That comparison shows potential value, not a proof that rejecting every new object is a good general policy.
Eviction quality cannot compensate for every harmful admission. The correct tradeoff depends on future reuse and changing popularity.
The living notebook
Explore the model
Interactive lab · cache
One eviction policy, two capacity budgets
Reset, set C=4, s=4: predict twelve hits out of sixteen. Reset and re-set C=3, s=4: predict zero hits. This capped 4096-request empty-start LRU model always admits misses; it cannot compare admission algorithms, variable object sizes, or Redis policies.
- Hits / requests
- 0 / 16
- Hit rate
- 0%
- Address-stream period
- 16 addresses
A fully associative LRU cache starts empty and requests address (i·s) mod 16. Hits move an address to the newest position; misses evict the oldest when full. Playback starts with 16 requests and adds two per second, capped at 4096 explicitly simulated requests. This toy stream is not a processor-cache benchmark. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.
Look forState the policy and resource budget before interpreting a hit-rate comparison.
Pause & reason
Check your understanding
Your turn
Try it yourself
A cache serves ninety 1 KiB hits and ten 100 KiB misses. Compute request hit ratio and requested-byte hit ratio, ignoring metadata. Explain why choosing a policy solely by the first number can mislead.
Take a moment to make a prediction or work it out on paper. Open a hint when you need one.
Give me a hint
Total requested bytes include every miss payload even though it was not served from cache.
Compare with the solution
Request hit ratio is 90/100=90%. Hit bytes are 90 KiB while total requested bytes are 90+1,000=1,090 KiB, giving 90/1,090≈8.26% byte hits. The remaining large misses dominate payload traffic and may dominate origin cost. A policy objective should reflect the actual bottleneck, freshness, and resource budget rather than maximizing only the most favorable ratio.
Keep the idea
Take it with you
- Admission decides whether to store; eviction decides what leaves.
- One-time scans can displace useful data under admit-everything policies.
- Compare policies at equal resource budgets with metrics that reflect actual avoided work.
Read further
Original sources and deeper explanations to follow at your own pace.
Optional progress, saved in this browser when available. Revisit any lesson at any time.
Lesson not marked complete.