Engineering / advanced
Construct auditable model training corpora
Turn raw collections into traceable training mixtures through deduplication, filtering, sampling, contamination checks, and documented use constraints.
Your route through engineering
Follow the path or begin with a question. Every lesson is open.
Your place, at your pace. Progress is optional and saved only in this browser when available.
Read / explore / reason
Inside this module
Module 01 advanced
Construct auditable model training corpora
Turn raw collections into traceable training mixtures through deduplication, filtering, sampling, contamination checks, and documented use constraints.
- 01
A training source needs provenance and a documented use decision
Build a source inventory that separates availability, provenance, use constraints, and project authorization without inferring rights from public access or a dataset label.
25 min - 02
Canonical text is a versioned representation choice
Separate raw capture, decoding, extraction, Unicode normalization, and content identity while preserving structure needed for code, mathematics, and reproducibility.
25 min - 03
Duplicate detection needs a representation, a similarity rule, and a retention policy
Separate exact identity from shingle resemblance, derive MinHash candidate behavior, and preserve cluster lineage without claiming perfect semantic deduplication.
27 min - 04
Language labels change the corpus mixture when their errors are uneven
Treat language identification as a versioned prediction with domain-specific uncertainty, abstention, and measurable retention effects on multilingual coverage.
26 min - 05
A quality filter needs a target, an audit, and a threshold tradeoff
Define a task-relevant quality rubric, measure accepted and rejected material, and separate score calibration from ranking, coverage, and downstream model gains.
27 min - 06
Redaction is a measured detection pipeline with residual risk
Define sensitive-data categories, audit detection and span replacement, and distinguish measured recall from a guarantee that a corpus cannot expose private information.
28 min - 07
An overlap test measures a threat model, not the absence of contamination
Search for evaluation leakage with explicit representations and thresholds, preserve protected holdouts, and distinguish observed overlap from its causal effect on scores.
28 min - 08
Mixture weights allocate training exposure rather than create new information
Distinguish corpus size from sampling probability, calculate repeated exposure under a fixed token budget, and evaluate reweighting without claiming a universally optimal mixture.
27 min - 09
Generated examples need provenance and evidence beyond their judge's approval
Trace generation and filtering decisions, quantify retained yield, and evaluate synthetic data with independent checks while keeping recursive-training claims scoped to their assumptions.
28 min - 10
An immutable release identifies what ran and how affected data can be retired
Build reproducible corpus manifests and dependency records, then distinguish historical identity, future-use retirement, physical deletion, and changes to trained models.
29 min