Engineering / advanced
Reason about deep learning architectures
Study architectural choices through mechanisms, computational constraints, and controlled comparisons.
Your route through engineering
Follow the path or begin with a question. Every lesson is open.
Your place, at your pace. Progress is optional and saved only in this browser when available.
Read / explore / reason
Inside this module
Module 01 advanced
Reason about deep learning architectures
Study architectural choices through mechanisms, computational constraints, and controlled comparisons.
- 01
Residual networks and normalization
Trace identity paths, normalization axes, and activation scales through a deep computation graph.
32 min - 02
Convolution and inductive bias
Derive convolution shapes and costs while separating locality, weight sharing, and translation behavior.
31 min - 03
Sequence models and state spaces
Derive recurrent state updates, convolution kernels, and the limits of fixed-memory sequence summaries.
35 min - 04
Positional representations and rotary embeddings
Derive how coordinate rotations make an attention score depend on relative token position.
34 min - 05
Sparse mixtures of experts
Follow routed tokens through expert capacity, communication, and load-balancing constraints.
35 min - 06
Representation learning and contrastive losses
Derive a contrastive objective and examine the assumptions hidden in positive and negative examples.
34 min - 07
Vision transformers and multimodal interfaces
Trace patch shapes and distinguish contrastive dual encoders from cross-attention fusion.
35 min - 08
Diffusion and score models
Connect a known noising process to learned denoising, scores, and iterative generation costs.
36 min - 09
Parameter-efficient adaptation
Derive low-rank update dimensions and distinguish trainable-state savings from total model memory.
33 min - 10
Architecture evaluation and ablation
Turn architecture comparisons into controlled experiments with explicit estimands, budgets, and uncertainty.
34 min