research30 min studyLesson 814 in Engineering

Align clip support, event labels, and the prediction cutoff

Define half-open source-time intervals, overlap policies, temporal IoU, and feature availability without treating offline annotations as already observable inputs.

On this page

What you’ll understand

  • Calculate overlap and temporal IoU using consistent source-time coordinates.
  • Distinguish a clip-label policy from event-localization evaluation.
  • Separate future outcome labels from forbidden future input features.

A clip's label depends on the question being asked. A video-level action tag, a per-frame state, a temporal event interval, and an online prediction target are not interchangeable. Write the source-time support and label rule before choosing clips. An array of sixteen images can cover half a second or several seconds, and the same action can occupy different fractions of those spans. Alignment errors can therefore produce a clean-looking supervised dataset whose labels refer to a different temporal question.

State interval conventions and overlap rules

Use one timebase and origin for the clip and annotation, then specify endpoints. Half-open intervals [s,e) avoid counting one boundary instant twice when adjacent segments meet. The overlap duration is the positive part of the earlier end minus the later start. A clip-positive rule might require any overlap, a minimum fraction of clip duration, a center point inside the event, or another task-specific criterion. These rules can produce different targets for the same annotation and must not be silently mixed.

Localization evaluation asks another question: how closely does a predicted event interval match the annotated interval? Temporal intersection-over-union compares intersection with union, while benchmark average precision additionally depends on confidence ordering, class, and matching conventions. Do not relabel a clip's overlap fraction as temporal IoU or infer benchmark AP from one interval. Preserve the original event identity and uncertainty when annotations are approximate, so preprocessing does not create artificial millisecond precision from coarse human labels.

I=max⁡ ⁣(0,min⁡(ec,ey)−max⁡(sc,sy)),tIoU=I(ec−sc)+(ey−sy)−II=\max\!\left(0,\min(e_c,e_y)-\max(s_c,s_y)\right),\qquad {\rm tIoU}=\frac{I}{(e_c-s_c)+(e_y-s_y)-I}
Clip and event intervals have positive finite duration in the same seconds-based coordinate system. I is seconds and temporal IoU is dimensionless. The formula alone does not define positive clip labels, confidence ranking, one-to-one detection matching, or annotation uncertainty.↔ Scroll wide equations horizontally.

Keep the target horizon separate from feature support

For an online decision at a declared cutoff, input features must satisfy the task's causal availability rule. A future outcome can be a legitimate training target, such as an action beginning in the next second, while future images must not be included as already observed inputs. Offline annotations can be created later to identify that outcome. Their later creation is not inherently leakage; using annotation-derived future information as a model input would be. Distinguish label knowledge during training from what the deployed predictor may observe.

Presentation time alone is insufficient for availability because decoding can require reordered references and transport can delay frames. Define a mapping between media position and the observed decision clock, then record when each feature becomes usable. An offline clip model that includes centered context or bidirectional temporal attention is not automatically an online model. Evaluation should either allow the stated lookahead and count its latency or enforce a causal feature construction and compare against that different contract.

F(t)={j:aj≤t}\mathcal F(t)=\{j:a_j\leq t\}
a_j is observed feature-ready time in the same elapsed-time domain as decision time t. This set specifies availability, not just media PTS. Label targets may concern a declared future horizon; they do not authorize including unavailable future features in F(t). Clock mappings and processing latency must be established separately.↔ Scroll wide equations horizontally.

Audit the clip manifest before training

Retain source video and event IDs, source intervals, chosen frame PTS values, target cutoff and horizon, label rule, and missing-data policy. Include clips that straddle action boundaries, contain multiple actions, or have no valid sampled frame in an intended subinterval. Check whether a clip was selected using label information in a way that changes the evaluated population. Selecting only obvious positive moments can be a valid training policy but does not represent an unfiltered online stream without a corresponding evaluation design.

The probability lab isolates a hypothetical rate of clips satisfying a declared rule. Reset, set both p=0.5 and n=20, predict ten expected qualifying clips, then Play or Step. Reset and set both p=0.75,n=20 for fifteen. These exact binomial counts assume independent identical qualifications and are not sampled video clips. The instrument computes no interval overlap, event label, feature-ready time, or temporal correlation; it illustrates why a rule's selected population should be counted rather than treated as the whole source stream.

Separate overlap fraction from temporal IoU

A clip covers [2.0,2.8) s and an event covers [2.2,3.0) s in the same source timeline. A fictional clip-label rule requires at least half the clip duration to overlap the event. Use exact stated endpoints.

  1. The intersection is [2.2,2.8), with duration 0.6 s. Both clip and event last 0.8 s.

  2. The clip-overlap fraction is 0.6/0.8=0.75, so the declared half-clip rule labels this clip positive.

  3. The union lasts 0.8+0.8−0.6=1.0 s, giving temporal IoU 0.6. The positive label's fraction 0.75 and localization score 0.6 are different quantities.

The result

The clip is positive under the specified rule and has temporal IoU 0.6 with the event. Neither number defines an online feature cutoff or benchmark average precision by itself.

Explore the model

Interactive lab

Interactive lab · probability

Count qualification under a declared clip rule

Fresh Reset → set both p=0.5,n=20 predicts mean count 10; Reset → set both p=0.75,n=20 predicts 15. Play or Step highlights exact binomial outcomes. The model has no clips, temporal IoU, feature availability, or correlated frames.

Count qualification under a declared clip ruleThese are exact binomial probabilities for independent trials with one fixed success probability. Playback tours the distribution; it does not resample outcomes. Current readings: Mean · np: 5; Variance · np(1−p): 2.5; Total probability: 1. All parameters are adjustable with the labeled controls below.03581000.070.150.220.3Successes kP(X = k)k = 0 · P = 9.77e-4
Exact probabilityHighlighted outcomeMean np
Mean · np
5
Variance · np(1−p)
2.5
Total probability
1
t = 0 s

These are exact binomial probabilities for independent trials with one fixed success probability. Playback tours the distribution; it does not resample outcomes. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forA label-selection policy defines a population that should remain visible in the dataset and evaluation record.

Check your understanding

An online model predicts whether an event will begin in the next second. Which training arrangement can be valid?

Try it yourself

A predictor makes a decision at elapsed time 5.0 s. A frame's mapped media position is 4.9 s, but transport and decoding make it feature-ready at 5.2 s. May that feature be included in a strict decision-at-five fixture, and why?

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

Apply the availability-time predicate rather than comparing only media position with the decision time.

Compare with the solution

No. Its feature-ready time 5.2 s exceeds the 5.0-second cutoff, even though its media position is earlier. Including it would give the fixture information unavailable at the declared decision. The system may instead wait and report a later decision with the added latency, or predict using the available feature set. These are different operational contracts and should not be compared as though they had identical timeliness.

Take it with you

  • Clip overlap rules and event-localization metrics answer different questions.
  • Future outcome targets can be valid while future unavailable input features are not.
  • Preserve source intervals, sampled PTS, label rules, and causal cutoffs in the clip manifest.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.