Align clip support, event labels, and the prediction cutoff
Define half-open source-time intervals, overlap policies, temporal IoU, and feature availability without treating offline annotations as already observable inputs.
On this page
The question for today
What you’ll understand
- Calculate overlap and temporal IoU using consistent source-time coordinates.
- Distinguish a clip-label policy from event-localization evaluation.
- Separate future outcome labels from forbidden future input features.
A clip's label depends on the question being asked. A video-level action tag, a per-frame state, a temporal event interval, and an online prediction target are not interchangeable. Write the source-time support and label rule before choosing clips. An array of sixteen images can cover half a second or several seconds, and the same action can occupy different fractions of those spans. Alignment errors can therefore produce a clean-looking supervised dataset whose labels refer to a different temporal question.
State interval conventions and overlap rules
Use one timebase and origin for the clip and annotation, then specify endpoints. Half-open intervals [s,e) avoid counting one boundary instant twice when adjacent segments meet. The overlap duration is the positive part of the earlier end minus the later start. A clip-positive rule might require any overlap, a minimum fraction of clip duration, a center point inside the event, or another task-specific criterion. These rules can produce different targets for the same annotation and must not be silently mixed.
Localization evaluation asks another question: how closely does a predicted event interval match the annotated interval? Temporal intersection-over-union compares intersection with union, while benchmark average precision additionally depends on confidence ordering, class, and matching conventions. Do not relabel a clip's overlap fraction as temporal IoU or infer benchmark AP from one interval. Preserve the original event identity and uncertainty when annotations are approximate, so preprocessing does not create artificial millisecond precision from coarse human labels.
Keep the target horizon separate from feature support
For an online decision at a declared cutoff, input features must satisfy the task's causal availability rule. A future outcome can be a legitimate training target, such as an action beginning in the next second, while future images must not be included as already observed inputs. Offline annotations can be created later to identify that outcome. Their later creation is not inherently leakage; using annotation-derived future information as a model input would be. Distinguish label knowledge during training from what the deployed predictor may observe.
Presentation time alone is insufficient for availability because decoding can require reordered references and transport can delay frames. Define a mapping between media position and the observed decision clock, then record when each feature becomes usable. An offline clip model that includes centered context or bidirectional temporal attention is not automatically an online model. Evaluation should either allow the stated lookahead and count its latency or enforce a causal feature construction and compare against that different contract.
Audit the clip manifest before training
Retain source video and event IDs, source intervals, chosen frame PTS values, target cutoff and horizon, label rule, and missing-data policy. Include clips that straddle action boundaries, contain multiple actions, or have no valid sampled frame in an intended subinterval. Check whether a clip was selected using label information in a way that changes the evaluated population. Selecting only obvious positive moments can be a valid training policy but does not represent an unfiltered online stream without a corresponding evaluation design.
The probability lab isolates a hypothetical rate of clips satisfying a declared rule. Reset, set both p=0.5 and n=20, predict ten expected qualifying clips, then Play or Step. Reset and set both p=0.75,n=20 for fifteen. These exact binomial counts assume independent identical qualifications and are not sampled video clips. The instrument computes no interval overlap, event label, feature-ready time, or temporal correlation; it illustrates why a rule's selected population should be counted rather than treated as the whole source stream.
Work it through
Separate overlap fraction from temporal IoU
A clip covers [2.0,2.8) s and an event covers [2.2,3.0) s in the same source timeline. A fictional clip-label rule requires at least half the clip duration to overlap the event. Use exact stated endpoints.
The intersection is [2.2,2.8), with duration 0.6 s. Both clip and event last 0.8 s.
The clip-overlap fraction is 0.6/0.8=0.75, so the declared half-clip rule labels this clip positive.
The union lasts 0.8+0.8−0.6=1.0 s, giving temporal IoU 0.6. The positive label's fraction 0.75 and localization score 0.6 are different quantities.
The clip is positive under the specified rule and has temporal IoU 0.6 with the event. Neither number defines an online feature cutoff or benchmark average precision by itself.
The living notebook
Explore the model
Interactive lab · probability
Count qualification under a declared clip rule
Fresh Reset → set both p=0.5,n=20 predicts mean count 10; Reset → set both p=0.75,n=20 predicts 15. Play or Step highlights exact binomial outcomes. The model has no clips, temporal IoU, feature availability, or correlated frames.
- Mean · np
- 5
- Variance · np(1−p)
- 2.5
- Total probability
- 1
These are exact binomial probabilities for independent trials with one fixed success probability. Playback tours the distribution; it does not resample outcomes. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.
Look forA label-selection policy defines a population that should remain visible in the dataset and evaluation record.
Pause & reason
Check your understanding
Your turn
Try it yourself
A predictor makes a decision at elapsed time 5.0 s. A frame's mapped media position is 4.9 s, but transport and decoding make it feature-ready at 5.2 s. May that feature be included in a strict decision-at-five fixture, and why?
Take a moment to make a prediction or work it out on paper. Open a hint when you need one.
Give me a hint
Apply the availability-time predicate rather than comparing only media position with the decision time.
Compare with the solution
No. Its feature-ready time 5.2 s exceeds the 5.0-second cutoff, even though its media position is earlier. Including it would give the fixture information unavailable at the declared decision. The system may instead wait and report a later decision with the added latency, or predict using the available feature set. These are different operational contracts and should not be compared as though they had identical timeliness.
Keep the idea
Take it with you
- Clip overlap rules and event-localization metrics answer different questions.
- Future outcome targets can be valid while future unavailable input features are not.
- Preserve source intervals, sampled PTS, label rules, and causal cutoffs in the clip manifest.
Read further
Original sources and deeper explanations to follow at your own pace.
Optional progress, saved in this browser when available. Revisit any lesson at any time.
Lesson not marked complete.