advanced29 min studyLesson 804 in Engineering

CTC sums alignment paths whose blanks and repeats collapse in a defined order

Work through CTC path probabilities and repeated labels while separating the alignment objective, decoder state, and causal encoder needed for streaming.

On this page

What you’ll understand

  • Collapse a CTC path without deleting legitimate repeated output labels.
  • Calculate a short target's probability by summing compatible paths.
  • Explain why CTC loss alone does not make an encoder causal or a partial transcript final.

Connectionist temporal classification lets a model learn sequence labels without one manually fixed label-to-frame alignment. It does this by summing many paths through frame-level labels and a special blank symbol. The collapse operation is easy to misapply: blanks separate repeated output labels, so removing blanks before merging repeats can change the transcript. Streaming adds another contract because the model and decoder must retain compatible context and decide when a provisional prefix is stable enough for the application to publish.

A blank is an alignment symbol, not a printed character

For each encoder time step, a CTC model defines a distribution over its label alphabet plus blank. Collapse first merges adjacent repetitions of the same path label, then removes blanks. Thus a,a,blank,a,b,b becomes a,a,b, preserving two output a labels separated in the path. Repeated output labels need separating blank positions in a minimal alignment. These labels may be characters or other tokens rather than words, and encoder steps can represent subsampled feature frames rather than individual waveform samples.

The original Graves and colleagues paper defines the path-summing objective and its dynamic program. The target probability includes every compatible alignment, not only the most likely framewise path. A greedy path decoder is convenient, but its collapsed output need not maximize the summed target probability. A prefix beam can retain several output hypotheses and combine equivalent alignment contributions. Keep blank-ending and nonblank-ending contributions distinct where the recurrence requires them; otherwise repeated-label handling can silently change even when the top-level transcript looks plausible on easy examples.

P(y∣x)=∑π:B(π)=y∏t=1Tpt(πt∣x)P(y\mid x)=\sum_{\pi:\mathcal B(\pi)=y}\prod_{t=1}^{T}p_t(\pi_t\mid x)
T is encoder time steps, pi a length-T path over labels plus blank, and B the merge-adjacent-repeats-then-remove-blanks map. The product is the CTC conditional factorization for fixed input x. Probabilities are dimensionless; summing paths differs from taking one best path.↔ Scroll wide equations horizontally.

Alignment freedom does not supply future-free computation

CTC can be used with a causal or bounded-lookahead encoder, but the loss itself does not restrict the encoder's receptive field. An offline bidirectional model may use the entire recording to predict an early frame and cannot become a genuine live recognizer merely by receiving audio in smaller arrays. Specify feature centering, encoder right context, subsampling, and chunk cache state. Emformer is a primary example of explicitly designed streaming context and memory; its latency settings are architectural and execution choices rather than consequences of the letters CTC.

A partial transcript can change when new evidence arrives, even when the acoustic computation is causal. Define whether the application receives unstable hypotheses, a committed prefix, or only a final result. A token spike is not automatically a calibrated word boundary or a definitive end of turn. Across chunks, carry the decoder state needed to distinguish a repeated label from a continuation of the same path run. Independently collapsing each chunk and concatenating the resulting strings can duplicate or delete labels at the boundary.

B(a,a,⊥,a,b,b)=(a,a,b),B(a,⊥,a)=(a,a)\mathcal B(a,a,\bot,a,b,b)=(a,a,b),\qquad \mathcal B(a,\bot,a)=(a,a)
Bottom denotes the CTC blank, not an ordinary output token. These finite examples apply the stated collapse order. Removing blanks first and then merging all adjacent equal labels would incorrectly turn the repeated output a into a single label.↔ Scroll wide equations horizontally.

Test decoder state and streaming latency separately

Use tiny enumerated paths to verify collapse and dynamic-programming logic, then compare chunked recognition with a reference under the same allowed context. Test repeated characters, long blank runs, boundary-spanning label runs, and empty outputs. Record whether scores are summed in probability space or combined with stable log-sum-exp arithmetic. Beam pruning and language-model additions change the search contract and may change scores' interpretation. Evaluate word error, partial revision rate, and emission delay separately; a low final error rate does not establish low live interaction latency.

The probability lab counts label-like successes but discards their order. Reset, explicitly set BOTH p=0.6 and n=3, predict mean count 1.8, then Play or Step. Reset and re-set BOTH p=0.8 and n=3; predict 2.4. Edits retain t and playback highlights exact binomial counts. There is no CTC decoder or blank alphabet. Two three-step paths with two a labels, a,a,blank and a,blank,a, have the same count yet collapse differently. This limitation shows why aggregate label counts cannot replace an ordered alignment calculation.

Enumerate a three-step CTC distribution

Use three encoder steps with alphabet {a,blank}. At every step p(a)=0.6 and p(blank)=0.4 under the CTC factorization. Calculate probabilities of the empty output, output aa, and output a.

  1. Only blank,blank,blank collapses to empty, with probability 0.4³=0.064. Only a,blank,a yields aa in three steps, with probability 0.6×0.4×0.6=0.144.

  2. The other six of eight paths collapse to a. Their total probability is 1−0.064−0.144=0.792, using the exhaustive two-symbol three-step path space.

  3. The greedy path a,a,a has probability 0.6³=0.216 and collapses to a, but 0.216 is not the output-a probability. The target probability sums all compatible alignments; this example's greedy label happens to agree with the most probable output.

The result

Path probability, collapsed output, and summed target probability are distinct quantities even in a tiny CTC model.

Explore the model

Interactive lab

Interactive lab · probability

A count distribution cannot represent ordered CTC collapse

For EACH run Reset, set BOTH p and n, predict, then Play or Step. At n=3 compare p=0.6, mean 1.8, with p=0.8, mean 2.4. Edits retain t. This exact binomial count model has no ordered path, blank state, collapse map, encoder, or CTC beam search.

A count distribution cannot represent ordered CTC collapseThese are exact binomial probabilities for independent trials with one fixed success probability. Playback tours the distribution; it does not resample outcomes. Current readings: Mean · np: 5; Variance · np(1−p): 2.5; Total probability: 1. All parameters are adjustable with the labeled controls below.03581000.070.150.220.3Successes kP(X = k)k = 0 · P = 9.77e-4
Exact probabilityHighlighted outcomeMean np
Mean · np
5
Variance · np(1−p)
2.5
Total probability
1
t = 0 s

These are exact binomial probabilities for independent trials with one fixed success probability. Playback tours the distribution; it does not resample outcomes. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forCTC requires ordered path state; identical label counts can correspond to different collapsed transcripts.

Check your understanding

What is the CTC collapse of a,a,blank,a under the stated convention?

Try it yourself

For two encoder steps with p(a)=0.5 and p(blank)=0.5 at each step, calculate probabilities of empty output and output a. Can output aa occur in those two steps?

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

Enumerate four paths and remember that a,a is one repeated run.

Compare with the solution

The all-blank path has probability 0.25. The other three paths, a,a; a,blank; and blank,a, each have probability 0.25 and collapse to a, so P(a)=0.75. Output aa cannot occur in two steps because a separating blank would require a third position.

Take it with you

  • CTC collapse merges adjacent path repetitions before removing blanks.
  • Target probabilities sum compatible alignments rather than reuse one best-path probability.
  • Streaming needs a context-bounded encoder and persistent decoder state in addition to CTC training.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.