Engineering / advanced
Build streaming speech and audio systems
Connect sample clocks, framing, recognition alignment, synthesis, and real-time buffering to measurable audio quality and interaction latency.
Your route through engineering
Follow the path or begin with a question. Every lesson is open.
Your place, at your pace. Progress is optional and saved only in this browser when available.
Read / explore / reason
Inside this module
Module 01 advanced
Build streaming speech and audio systems
Connect sample clocks, framing, recognition alignment, synthesis, and real-time buffering to measurable audio quality and interaction latency.
- 01
Audio samples belong to a clock, not merely to a numbered array
Relate sample counts to physical time, quantify clock drift, and distinguish filtered rate conversion from relabeling, decimation, and offline boundary handling.
27 min - 02
Spectral features inherit the frame, window, FFT, and normalization contract
Calculate STFT dimensions and timing, distinguish bin spacing from resolving power, and normalize amplitudes without confusing spectra, power, and density.
28 min - 03
Voice activity is evidence for an endpoint policy, not the endpoint itself
Separate frame-level speech decisions from turn completion, quantify hangover delay, and evaluate false endpoints under real noise and conversational pauses.
27 min - 04
CTC sums alignment paths whose blanks and repeats collapse in a defined order
Work through CTC path probabilities and repeated labels while separating the alignment objective, decoder state, and causal encoder needed for streaming.
29 min - 05
A transducer carries both acoustic position and emitted-label history
Follow blank and label transitions in an RNN-T lattice, combine equivalent paths, and distinguish search limits from model probabilities and streaming context guarantees.
29 min - 06
Chunk context may overlap while committed sample ownership remains disjoint
Assign half-open core intervals, preserve filter and model state, and distinguish reused context from duplicated samples, frames, or recognition tokens.
28 min - 07
A jitter buffer schedules against a media clock while absorbing bounded arrival variation
Separate RTP timestamps, arrival variation, clock drift, and playout deadlines while measuring the latency and late-loss costs of buffer policy.
28 min - 08
Streaming synthesis commits audio before all future text is known
Separate text lookahead, acoustic and vocoder work, playout, and context state while evaluating prosody and continuity under incremental input.
28 min - 09
Interruption stops owned output while echo control follows what was actually rendered
Track generation identifiers, queued and device-resident audio, and aligned render references without confusing cancellation with already played sound or echo suppression with speech preservation.
29 min - 10
Evaluate normalized recognition errors and the complete timed interaction
Compute word error with a fixed scoring protocol, distinguish corpus weighting from cohort averages, and measure audible latency, revision, failure, and quality together.
29 min