Engineering / advanced
Serve ranking systems with feedback
Trace recommendation candidates through embeddings, ranking, constraints, feedback, and experimentation while making exposure bias explicit.
Your route through engineering
Follow the path or begin with a question. Every lesson is open.
Your place, at your pace. Progress is optional and saved only in this browser when available.
Read / explore / reason
Inside this module
Module 01 advanced
Serve ranking systems with feedback
Trace recommendation candidates through embeddings, ranking, constraints, feedback, and experimentation while making exposure bias explicit.
- 01
Candidate coverage sets a ceiling that a downstream ranker cannot repair
Trace retrieval unions, eligibility filters, and relevance coverage before attributing an end-to-end recommendation failure to the ranking model.
30 min - 02
Embedding serving moves indexed bytes across a placement and cache hierarchy
Account for row storage, request lookups, deduplication, cache misses, and sharding traffic before treating embedding serving as a dense compute problem.
31 min - 03
Two-tower retrieval requires matching representations and the exact search metric
Connect precomputed item vectors, query encoding, inner products, normalization, approximate search, and version compatibility.
32 min - 04
A ranking cascade changes both its workload and the population reaching later stages
Analyze candidate filtering, request escalation, conditional quality, and mixture latency without confusing cohort differences with treatment effects.
31 min - 05
A constrained slate needs a feasible exposure allocation, not only a weighted score
Separate prediction, utility, position exposure, hard eligibility, and aggregate constraints when constructing a recommendation slate.
33 min - 06
Observed clicks depend on exposure, position, and the policy that selected an item
State the identification and support assumptions behind propensity weighting instead of treating logged click rates as unbiased utility measurements.
34 min - 07
Exploration spends a defined reward budget to learn about eligible alternatives
Specify action probabilities, contextual reward assumptions, cold-start priors, and the comparator behind regret instead of promising universal benefit from random recommendations.
33 min - 08
Session state needs versions, ordering, and retry semantics across concurrent requests
Keep recommendation history, model artifacts, and feedback references coherent when tabs, devices, retries, and out-of-order updates share a session.
32 min - 09
Feedback needs stable identity, exposure attribution, and two separate notions of time
Prevent repeated delivery and late knowledge from changing recommendation labels silently while preserving distinct actions and the policy that exposed an item.
33 min - 10
Judge recommendation quality, completed tail latency, and cost on the same cohorts
Build a versioned evaluation contract that includes outcome meaning, request completion, timeouts, resource accounting, and population shifts.
34 min