advanced25 min studyLesson 681 in Engineering

Feature definitions and entity keys

Specify semantic identities, typed join keys, units, time boundaries, and versioned transformations before storing model inputs.

On this page

What you’ll understand

  • Define a feature contract beyond its column name and storage type.
  • Identify composite entity keys without ambiguous concatenation.
  • Distinguish an unavailable observation from a valid numerical zero.

A feature is a measured or computed property with an interpretation, not just a named number. The same field can mean a lifetime count, a ten-minute rate, or a delayed estimate, and each meaning leads to different model behavior. A feature system starts with a contract for entity identity, transformation, units, time, version, and missingness. Storage and retrieval should preserve that contract across training and serving.

An entity key identifies the intended subject

An entity can be a user, merchant, device, session, or a composite such as user within merchant category. Feast distinguishes an entity name from the physical join keys used to retrieve values. Include tenancy and domain where identifiers are not globally unique. Preserve types and canonical encodings: a string identifier with leading zeros may not equal an integer with the same displayed digits. Normalization rules must be shared by writers, historical joins, and online requests or values can silently attach to the wrong subject.

Naively concatenating composite fields can create collisions. The tuples (1,23) and (12,3) both become the string 123 if separators and lengths are omitted. Use an unambiguous typed encoding rather than relying on a delimiter that might itself occur in data. Hashing an already ambiguous string cannot recover lost distinctions. A key migration therefore changes a data contract, not merely a storage optimization. Plan how old and new requests resolve identity, and avoid accidentally mixing namespaces during a staged migration.

K=(tenant⁡,entity type⁡,typed join fields⁡)K=(\operatorname{tenant},\operatorname{entity\ type},\operatorname{typed\ join\ fields})
K is a logical composite identity. Its physical serialization must preserve field boundaries and types within the supported domain; this tuple notation does not guarantee injective encoding or collision-free hashing automatically.↔ Scroll wide equations horizontally.

Version the complete feature meaning

A definition should name the source and its version, selection predicates, deduplication rule, window boundaries, aggregation, units, output type, missing-value policy, and transformation revision. For a purchase rate, distinguish events from successful payments and define whether canceled purchases are retracted. State the timestamp that anchors the window and whether the upper boundary includes an event exactly at prediction time. A feature reference used by a model must resolve to the intended definition, even when another team publishes a similarly named field.

Schema compatibility is broader than numerical type compatibility. Changing cents to dollars or a count to a rate can leave the field represented as Float32 while completely changing its meaning. A model’s feature bundle should fix ordering and definitions instead of depending on whichever view is currently active. Feast feature services group features used by model versions, but application governance must still bind the intended transformation and source contracts. Publishing a definition does not itself compute historical values, complete a backfill, or prove that every online key is populated.

r(e,t)=N(e,[t−w,t))wr(e,t)=\frac{N(e,[t-w,t))}{w}
This stipulated rate counts qualifying events for entity e in a half-open window of w positive seconds. N is an integer count, so r is events/s. Event identity, deduplication, availability, and window membership must be defined separately.↔ Scroll wide equations horizontally.

Missingness and validation belong to the interface

A missing feature may mean no qualifying activity, no source record, an expired value, a failed lookup, or an unmaterialized key. These are not interchangeable states. Define which cases legitimately map to zero and which trigger a default, mask, fallback model, or explicit failure. Attach event and availability metadata when required to assess the observation. Validate ranges and units using domain knowledge; a schema parser cannot tell whether a plausible-looking value came from the wrong customer or a future correction.

The fixed sigmoid instrument illustrates why interpretation changes matter downstream. Reset, set both x=1 and w=1, predict σ(0.5)≈0.62246, then Play or Step. Separately Reset, set both x=2 and w=1, predict σ(1.5)≈0.81757, then Play or Step. The model computes only σ(wx−0.5); doubling a feature representation without adapting the consumer changes its result. This is not a feature-store lookup, trained predictor, or key validator. It provides a concrete consequence of changing an input’s scale while keeping its consumer rule unchanged.

A count and a rate have different units

An entity has thirty qualifying deduplicated purchases in a ten-minute half-open window. A model expects purchases per second, while an incorrectly configured producer returns the raw count.

  1. Convert the ten-minute duration to 600 seconds. The defined rate is 30/600=0.05 purchases per second.

  2. The raw count is 30 purchases in the window, not 30 purchases per second. Supplying 30 where 0.05 is expected creates a numerical factor of 600 under this example.

  3. Both values can fit the same floating-point schema. A correct contract and unit-sensitive test must distinguish them; a successful typed lookup does not prove semantic compatibility.

The result

Feature correctness depends on identity and meaning, including units and boundaries. Storage type alone is too weak an interface.

Explore the model

Interactive lab

Interactive lab · network

Input scaling changes the fixed downstream computation

Reset, set both x=1 and w=1, predict output σ(0.5)≈0.62246, then Play or Step. Separately Reset, set both x=2 and w=1, predict σ(1.5)≈0.81757, then Play or Step. This one fixed-bias sigmoid neither retrieves features nor validates entity keys or model quality.

Input scaling changes the fixed downstream computationOne inspectable sigmoid unit, σ(wx − 0.5). Playback traces the computation graph. There is no training, attention mechanism or hidden deep network. Current readings: Product · wx: 1.2; Logit · wx − 0.5: 0.7; Sigmoid output: 0.6682. All parameters are adjustable with the labeled controls below.One input. One weight. One sigmoid.x1.20× w1.20+ b0.70σ0.668output0.668w = 1.00b = −0.50σ(z) = 1 / (1 + exp(−z))
Computation graphSignal trace
Product · wx
1.2
Logit · wx − 0.5
0.7
Sigmoid output
0.6682
t = 0 s

One inspectable sigmoid unit, σ(wx − 0.5). Playback traces the computation graph. There is no training, attention mechanism or hidden deep network. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forKeep producer representation and consumer interpretation versioned together.

Check your understanding

A purchase count becomes purchases per minute but keeps the same Float32 field name. Which statement is correct?

Try it yourself

Two tenant-local users both have user_id=42, one in tenant 7 and one in tenant 9. How many distinct logical entity keys are required, and why is a key of only user_id insufficient?

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

Ask whether identifiers are globally unique across the declared domain.

Compare with the solution

Two distinct keys are required, for example typed tuples (7,42) and (9,42) in the same user namespace. A key containing only 42 aliases the tenants and can return another subject’s features. The physical encoding must also preserve tuple boundaries, types, and the intended entity namespace.

Take it with you

  • Define typed entity identity and unambiguous key encoding.
  • Version source, transformation, units, windows, missingness, and model bundle together.
  • Treat semantic validation as distinct from storage-type validation.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.