Feature freshness and TTL semantics
Distinguish observation age, availability delay, cache lifetime, source completeness, and feature-specific validity policies.
On this page
The question for today
What you’ll understand
- Compute source-event and cache ages independently.
- Explain why a cache hit or renewed expiration does not prove fresh source data.
- Interpret historical lookup TTL separately from backend online retention.
A fast successful lookup can return a stale observation. A cache entry can be newly written using old source data, and a frequently read key can remain resident while its upstream producer has stopped. Freshness therefore needs a definition tied to feature meaning and source progress. A generic TTL is one operational mechanism, not a universal proof that the model’s information is current.
Name the timestamp whose age matters
For a latest-observation feature, event age measures how long ago the selected observation occurred. Availability age measures how long ago that version became usable on the relevant path. Pipeline delay compares those two timestamps, while cache age starts when the cache entry was filled. They answer different questions. A profile attribute that changes rarely can remain valid for months; a market quote may be unusable after seconds. Define feature-specific acceptable age and behavior when timestamps are absent or clocks are inconsistent.
Aggregates need additional care. The timestamp on a window result may identify its window endpoint, not the age of the newest contributing event or the completeness of upstream ingestion. A zero count for a recent interval can be correct when there were no events, or wrong when the source stalled. Monitor source progress, ingestion lag, watermark assumptions, and expected update cadence alongside the value’s timestamp. Do not declare every unchanged value stale or every recently recomputed value informed by recent complete source data.
A TTL has a particular enforcement scope
A cache lifetime often expires an entry relative to insertion or refresh, while a data-validity rule can reject a feature relative to its event timestamp. Redis EXPIRE supplies key-expiration behavior; it does not inspect a value’s embedded source timestamp. Rewriting an old observation with a new expiration can therefore keep old information available. Physical deletion timing, read-time filtering, registry-cache refresh, and source retention are distinct policies. Check the exact backend contract instead of assuming every parameter named ttl operates at the same layer.
Feast’s historical feature-view TTL bounds how far back retrieval searches relative to each entity-row timestamp. That is different from measuring age relative to the current time when generating a dataset. Online validity and physical retention also depend on implementation and configuration; verify them for the selected provider and version. A historical TTL does not make an upstream stream healthy or certify an online serving-age objective. A request should apply the declared missing or stale fallback policy using the metadata and guarantees it actually has.
Measure freshness by feature and entity class
Track age distributions, missing timestamps, update failures, source progress, and stale/default responses at the point where models consume values. A global average can hide hot entities that update promptly and rare entities that are never materialized. Preserve the model and feature definition versions so an alert can identify which consumer policy was violated. Decide explicitly whether serving an old but interpretable value is preferable to a default or failure; that tradeoff belongs to the model’s tested contract rather than an accidental cache behavior.
The LRU instrument illustrates that residency and hits do not contain age information. Reset, set both C=4 and s=4, predict four misses and thirteen hits after seventeen requests, then Step once. Separately Reset, set both C=2 and s=4, predict seventeen misses after seventeen requests, then Step once. Reset starts with sixteen requests and Step adds one. This toy tracks addresses and recency only: it has no TTL or source timestamps. A high hit count therefore cannot be interpreted as fresh feature observations, just as an actual cache hit alone cannot establish upstream recency.
Work it through
A young cache entry can contain an old event
A request is at 12:30:00. The selected observation’s event time is 11:00:00, it became available at 12:29:00, and a cache entry was filled at 12:29:50 with a sixty-second insertion-based lifetime. Use one consistent clock.
The event age is ninety minutes, or 5,400 seconds. Availability age is sixty seconds. The observation became usable recently but describes a much older event.
The cache age is ten seconds and its stipulated expiration is 12:30:50, leaving fifty seconds of timer eligibility at the request.
A cache hit is therefore possible while the event is ninety minutes old. Whether that value is acceptable depends on the feature’s validity policy and source-progress evidence, not on the remaining cache timer alone.
Report the age of the relevant observation and the behavior of the chosen layer separately. A new write or cache hit cannot manufacture recent source knowledge.
The living notebook
Explore the model
Interactive lab · cache
A hit counter has no source-age information
Reset, set both C=4 and s=4, predict four misses and thirteen hits after seventeen requests, then Step once. Separately Reset, set both C=2 and s=4, predict seventeen misses after seventeen requests, then Step once. The fixed modulo-16 LRU model has no TTL, event timestamp, or upstream feature source.
- Hits / requests
- 0 / 16
- Hit rate
- 0%
- Address-stream period
- 16 addresses
A fully associative LRU cache starts empty and requests address (i·s) mod 16. Hits move an address to the newest position; misses evict the oldest when full. Playback starts with 16 requests and adds two per second, capped at 4096 explicitly simulated requests. This toy stream is not a processor-cache benchmark. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.
Look forSeparate cache residency from feature validity and source freshness.
Pause & reason
Check your understanding
Your turn
Try it yourself
At 15:00, an observation has event time 14:50 and was cached at 14:59 with a five-minute insertion TTL. Compute event age and scheduled cache expiration.
Take a moment to make a prediction or work it out on paper. Open a hint when you need one.
Give me a hint
Use the event timestamp for one calculation and the fill timestamp for the other.
Compare with the solution
The event is ten minutes old, and the stipulated cache expiration is 15:04. At 15:00 the cache timer has four minutes remaining, but that does not reduce the event age. A separate feature policy decides whether a ten-minute-old observation is valid and whether source progress supports trusting it.
Keep the idea
Take it with you
- Define age using the observation and path that matter to the model.
- Separate historical lookback, cache expiry, physical retention, and source health.
- Monitor stale and missing responses at actual model consumption.
Read further
Original sources and deeper explanations to follow at your own pace.
Optional progress, saved in this browser when available. Revisit any lesson at any time.
Lesson not marked complete.