advanced24 min studyLesson 322 in Engineering

Admission is not eviction

Separate deciding what enters a cache from deciding what leaves under memory pressure.

On this page

What you’ll understand

  • Distinguish admission, replacement, expiration, and memory allocation.
  • Explain how one-time scans can displace valuable resident data.
  • Compare policies using request, byte, and origin-cost objectives.

A miss does not have to become a stored entry. Admission decides whether a candidate deserves cache space; eviction decides what existing data gives up that space. Keeping these decisions separate exposes why a cache can perform badly despite a plausible replacement rule. A stream of objects used once can push out a repeatedly useful working set unless the policy considers the value of admitting them.

Four decisions hide behind a set operation

An application may consider a response cacheable, an admission policy may reject it, a replacement policy may choose a victim, and an allocator may still lack room for its size class. Expiration separately determines whether an entry remains eligible for reuse by age. None of these is equivalent to the others. A miss counter cannot identify which decision caused absence, and an eviction count does not tell whether rejecting the incoming object would have been better.

Pure LRU favors recent access, while frequency-based admission can compare a candidate's estimated recent popularity with that of a proposed victim. Estimates can be approximate and aged so old popularity does not dominate forever. A policy tuned for stationary traffic can struggle after a workload shift. Avoid claiming one algorithm universally wins: the arrival distribution, object sizes, concurrency costs, and value of avoiding each origin fetch determine the useful comparison.

Vi≈ficisiV_i\approx\frac{f_i c_i}{s_i}
This illustrative value density uses expected reuse frequency fi in requests/s, avoided cost ci in seconds/request, and object size si in bytes. Its units are avoided origin-seconds per second per byte. It is a planning heuristic, not the TinyLFU algorithm or a universal optimal policy.↔ Scroll wide equations horizontally.

A scan can consume the whole budget

Suppose a cache holds a hot set that fits exactly, then a one-time scan accesses many new objects. An admit-everything LRU policy can evict the entire hot set while recording no future benefit from the scan entries. Rejecting some one-hit candidates can protect reuse, but a genuinely new popular object must eventually gain admission. Windowed or adaptive designs balance those pressures. Replay a trace with phase changes rather than optimizing only the final steady segment.

Object sizes matter as well as counts. One large object can displace several smaller valuable ones, and a high request hit rate can coexist with poor byte savings or expensive remaining misses. Metadata, allocator overhead, and policy bookkeeping consume memory and CPU. Compare actual resident bytes and origin work at the same total resource budget. A policy with a slightly higher simulated hit ratio may lose in production if its coordination cost overwhelms the saved fetches.

Hrequest=NhitNrequest,Hbyte=DhitDrequestedH_{\rm request}=\frac{N_{\rm hit}}{N_{\rm request}},\qquad H_{\rm byte}=\frac{D_{\rm hit}}{D_{\rm requested}}
Both ratios are dimensionless under compatible definitions. D counts requested object bytes, not network wire traffic; varying object sizes can make byte and request hit ratios diverge.↔ Scroll wide equations horizontally.

Use the lab as a baseline, not a policy tournament

Separate trace replay from live outcome measurement. A replay can estimate a defined policy's decisions for a recorded stream, but the trace may change when latency or admission changes. Warmup, expiration, invalidations, and concurrent requests need explicit treatment. In a live comparison retain correctness and freshness checks, observe origin cost, and sample policy CPU overhead. Do not compare a count-bounded simulation with a byte-bounded production cache as if their capacities were identical.

The cache lab implements only fully associative LRU and admits every miss. Reset and set both C=4, s=4, predict twelve hits in sixteen initial requests, then Step. Reset and re-set C=3, s=4; predict zero hits because the four-address cycle exceeds three entries. Its address stream is (i×s) mod 16 and capped at 4096 requests. There is no admission switch, variable size, TTL, or TinyLFU estimator. Explain how rejecting candidates would require a different trace and policy state rather than pretending a slider selects a new algorithm.

A four-object scan displaces a hot set

A four-entry LRU cache initially holds equally sized hot objects A, B, C, D. A one-time scan requests E, F, G, H, each missing and admitted, then requests A. Assume no expiration or concurrent accesses.

  1. Each new scan object requires one victim. After E, F, G, H are admitted, all four original hot objects have been displaced regardless of their initial recency order.

  2. The next access to A misses, making five misses over these five requests. The four scan fills produced no reuse within this declared interval.

  3. A hypothetical admission rule rejecting E through H would preserve A, B, C, D and make the final A access hit. That comparison shows potential value, not a proof that rejecting every new object is a good general policy.

The result

Eviction quality cannot compensate for every harmful admission. The correct tradeoff depends on future reuse and changing popularity.

Explore the model

Interactive lab

Interactive lab · cache

One eviction policy, two capacity budgets

Reset, set C=4, s=4: predict twelve hits out of sixteen. Reset and re-set C=3, s=4: predict zero hits. This capped 4096-request empty-start LRU model always admits misses; it cannot compare admission algorithms, variable object sizes, or Redis policies.

One eviction policy, two capacity budgetsA fully associative LRU cache starts empty and requests address (i·s) mod 16. Hits move an address to the newest position; misses evict the oldest when full. Playback starts with 16 requests and adds two per second, capped at 4096 explicitly simulated requests. This toy stream is not a processor-cache benchmark. Current readings: Hits / requests: 0 / 16; Hit rate: 0%; Address-stream period: 16 addresses. All parameters are adjustable with the labeled controls below.Fully associative LRU cache after 16 deterministic requests: 0 hits and 16 misses.Most recent 16 requestsaddress = i × 3 mod 160M3M6M9M12M15M2M5M8M11M14M1M4M7M10M13MCurrent cache order · oldest → newest471013Latest address 13 · miss: inserted as newestToy address stream · period 16 · starts empty
Cached addressMiss / evictionMost recent request
Hits / requests
0 / 16
Hit rate
0%
Address-stream period
16 addresses
t = 0 s

A fully associative LRU cache starts empty and requests address (i·s) mod 16. Hits move an address to the newest position; misses evict the oldest when full. Playback starts with 16 requests and adds two per second, capped at 4096 explicitly simulated requests. This toy stream is not a processor-cache benchmark. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forState the policy and resource budget before interpreting a hit-rate comparison.

Check your understanding

Which decision is specifically admission?

Try it yourself

A cache serves ninety 1 KiB hits and ten 100 KiB misses. Compute request hit ratio and requested-byte hit ratio, ignoring metadata. Explain why choosing a policy solely by the first number can mislead.

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

Total requested bytes include every miss payload even though it was not served from cache.

Compare with the solution

Request hit ratio is 90/100=90%. Hit bytes are 90 KiB while total requested bytes are 90+1,000=1,090 KiB, giving 90/1,090≈8.26% byte hits. The remaining large misses dominate payload traffic and may dominate origin cost. A policy objective should reflect the actual bottleneck, freshness, and resource budget rather than maximizing only the most favorable ratio.

Take it with you

  • Admission decides whether to store; eviction decides what leaves.
  • One-time scans can displace useful data under admit-everything policies.
  • Compare policies at equal resource budgets with metrics that reflect actual avoided work.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.