Engineering / advanced
Connect retrieved evidence to generation
Engineer ingestion, chunking, evidence selection, grounded output, access controls, and diagnosis across the complete retrieval-generation path.
Your route through engineering
Follow the path or begin with a question. Every lesson is open.
Your place, at your pace. Progress is optional and saved only in this browser when available.
Read / explore / reason
Inside this module
Module 01 advanced
Connect retrieved evidence to generation
Engineer ingestion, chunking, evidence selection, grounded output, access controls, and diagnosis across the complete retrieval-generation path.
- 01
Chunk boundaries: preserve source identity while budgeting overlap
Define token-window boundaries, overlap amplification, structural cuts, and versioned source mappings before building a retrieval index.
25 min - 02
Hierarchical retrieval: expand a precise hit without losing its lineage
Separate child retrieval, parent expansion, recursive summaries, authorization boundaries, and context budgets.
25 min - 03
Query rewriting: improve the search question without inventing its answer
Distinguish intent-preserving rewrites, independent expansions, and dependent subquestions while bounding fanout, latency, and evidence drift.
25 min - 04
Retrieval access control: enforce permission before evidence reaches a consumer
Bind retrieval, expansion, caches, citations, and answer serving to authenticated identity and current source permissions.
26 min - 05
Context packing: budget the rendered evidence, not a pile of hits
Deduplicate by source-aware identity and choose a bounded evidence portfolio with explicit tokenization, ordering, and coverage assumptions.
25 min - 06
Citation validation: resolve a source span, then test what it supports
Track immutable evidence locators and separate citation presence, resolvability, entailment, coverage, authority, and access.
26 min - 07
Conflicts and abstention: measure when evidence is sufficient to answer
Resolve source scope and version conflicts, define abstention behavior, and evaluate calibrated risk versus coverage without treating model confidence as a guarantee.
26 min - 08
Retrieval caches: validate the model, corpus, and access dependencies
Distinguish embedding, retrieval, context, and answer caches, with version-aware keys and explicit freshness and revocation behavior.
26 min - 09
Error attribution: change one evidence stage with a controlled oracle
Separate corpus coverage, retrieval, expansion, packing, and generation failures using aligned labels and controlled evidence interventions.
26 min - 10
Corpus updates: compare evidence and answers against a versioned manifest
Track ingestion and policy changes, publish coherent retrieval generations, and detect hidden regressions with paired answer and citation tests.
27 min