Sample presentation time without promising all motion survives
Distinguish timestamp-based selection from frame strides and demonstrate temporal aliasing under a precisely stated uniform-sampling model.
On this page
The question for today
What you’ll understand
- Calculate an aliasing counterexample for a uniformly sampled sinusoid.
- State the conditions of a bandlimited sampling argument.
- Define a selection rule for variable-frame timestamps and track its error.
Temporal subsampling chooses which evidence reaches the model. Sparse frames can preserve a person's appearance while losing the order or frequency of a fast action. More frames may help a task, but there is no unconditional sampling rate that preserves every real motion. Define the presentation-time grid, source timestamp selection rule, and information assumptions before interpreting a temporal model's performance. A model cannot recover distinctions that the chosen observations made identical without additional prior assumptions.
Separate a time grid from a source-frame stride
For a verified constant-rate segment, taking every kth source frame corresponds to a fixed interval if timestamps and missing-frame policy support that claim. For variable-rate input, the same ordinal stride can produce unequal time gaps. A timestamp-driven sampler can choose the nearest frame, the latest frame not later than a target, or another declared interpolation rule. These are different policies. Nearest selection can choose future presentation content and therefore is not automatically suitable for a strict online feature cutoff.
Store the target times, chosen source PTS values, timestamp errors, repeated selections, and uncovered gaps. If several target times map to one source frame, the output array has repeated images rather than new temporal evidence. Interpolation can create synthetic frames but does not make them independently observed content. A fixed-size tensor is a useful model interface, yet its shape does not establish that the intended temporal span or event ordering was preserved.
Use the sampling theorem only within its domain
For an ideal continuous signal bandlimited below a known highest frequency, exact uniform samples at a strictly higher rate than twice that frequency support the usual reconstruction argument with ideal assumptions. Real video involves finite exposure, occlusion, compression, camera motion, and potentially non-bandlimited changes. An anti-alias filter before downsampling can reduce frequencies outside the chosen band, while also removing information. It does not restore distinctions already aliased in the captured source.
SlowFast explores different temporal rates for complementary pathways, and TimeSformer studies space-time attention choices. Their task results motivate architecture and sampling experiments rather than a theorem that all fast motion is preserved. Report recognition quality by action frequency, duration, and source frame cadence where relevant. A task may tolerate lost high-frequency detail, or may depend exactly on it. Distinguish empirical task sufficiency from lossless reconstruction of an arbitrary physical scene.
Test both the sampling policy and the task
Build a timestamp fixture with irregular intervals and a synthetic signal with known frequencies. Test the selected times independently from the learned model so an indexing error cannot be explained away as model weakness. Then evaluate real task performance across rate and clip-span changes under the same split. If the model accepts timestamps as features, verify their units and handling of repeated frames; supplying an ordinal position still does not tell it the actual gap.
The waves lab provides a known analytic component that can be inspected at reachable times. Reset, set both phase difference 0 degrees and amplitude ratio 0, then predict the value at x=0 after four Steps: −sin(2)≈−0.9093. Reset and set both phase 0,ratio 1 for −2sin(2)≈−1.8186. Step spacing is half a second. This fixed-frequency analytic instrument does not downsample a video, run an anti-alias filter, or demonstrate the separate seven-versus-three-hertz example; it keeps the observed function and sampling times explicit.
Work it through
Make two different temporal frequencies look identical
Sample x(t)=cos(2π·7t) and y(t)=cos(2π·3t) uniformly at 10 samples per second. Use exact ideal point samples, with t=n/10 and no noise or exposure averaging.
At n=0 both values equal one. At n=1, cos(1.4π)=cos(0.6π)≈−0.309017.
At n=2, cos(2.8π)=cos(1.2π)≈−0.809017. More generally, 7n/10=n−3n/10 and cosine is even and periodic, proving equality at every integer sample.
The sampled sequences cannot identify which of these two source signals occurred. Increasing the model's capacity cannot remove that ambiguity without another assumption or additional observations.
The different seven- and three-hertz signals share every ten-hertz point sample. This proves one information-loss mechanism under explicit sampling assumptions, not a universal motion-recovery claim.
The living notebook
Explore the model
Interactive lab · waves
Name the analytic signal and the inspected timestamps
Reset → set both phase 0°,ratio 0; at x=0 predict −sin(2)≈−0.9093 after four Steps. Reset → set both phase 0°,ratio 1 for −2sin(2)≈−1.8186. The fixed wave has no video sampler, anti-alias filter, or adjustable seven-hertz component.
- Sum at x = 0
- 0.866
- Resultant amplitude
- 1.732
- Phase difference
- 60°
Two waves share the same frequency and wave number. Playback advances t in sin(2x − t) + a sin(2x − t + phase). Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.
Look forA temporal observation needs a known signal model and explicit times before supporting a reconstruction claim.
Pause & reason
Check your understanding
Your turn
Try it yourself
Available source times are [0.94,0.98,1.07,1.11] s and the target is 1.05 s. Which frame does nearest-time selection choose, and which does a latest-not-after rule choose? Calculate each signed timestamp error.
Take a moment to make a prediction or work it out on paper. Open a hint when you need one.
Give me a hint
Compare absolute distances for nearest selection, then exclude future presentation times for the second rule.
Compare with the solution
Nearest selection chooses 1.07 s, with error +0.02 s. Latest-not-after chooses 0.98 s, with error −0.07 s. The first is closer in presentation time but uses later content than the target; the second obeys that presentation cutoff while having larger temporal error. A live task additionally needs decode-ready availability, so even the earlier PTS is not by itself proof it was usable at the decision instant.
Keep the idea
Take it with you
- Frame strides and timestamp grids coincide only under established cadence assumptions.
- Aliasing creates indistinguishable observations, and reconstruction claims need explicit signal conditions.
- Record selected timestamps, repeats, gaps, and task-specific quality when changing sampling.
Read further
Original sources and deeper explanations to follow at your own pace.
Optional progress, saved in this browser when available. Revisit any lesson at any time.
Lesson not marked complete.