research30 min studyLesson 813 in Engineering

Sample presentation time without promising all motion survives

Distinguish timestamp-based selection from frame strides and demonstrate temporal aliasing under a precisely stated uniform-sampling model.

On this page

What you’ll understand

  • Calculate an aliasing counterexample for a uniformly sampled sinusoid.
  • State the conditions of a bandlimited sampling argument.
  • Define a selection rule for variable-frame timestamps and track its error.

Temporal subsampling chooses which evidence reaches the model. Sparse frames can preserve a person's appearance while losing the order or frequency of a fast action. More frames may help a task, but there is no unconditional sampling rate that preserves every real motion. Define the presentation-time grid, source timestamp selection rule, and information assumptions before interpreting a temporal model's performance. A model cannot recover distinctions that the chosen observations made identical without additional prior assumptions.

Separate a time grid from a source-frame stride

For a verified constant-rate segment, taking every kth source frame corresponds to a fixed interval if timestamps and missing-frame policy support that claim. For variable-rate input, the same ordinal stride can produce unequal time gaps. A timestamp-driven sampler can choose the nearest frame, the latest frame not later than a target, or another declared interpolation rule. These are different policies. Nearest selection can choose future presentation content and therefore is not automatically suitable for a strict online feature cutoff.

Store the target times, chosen source PTS values, timestamp errors, repeated selections, and uncovered gaps. If several target times map to one source frame, the output array has repeated images rather than new temporal evidence. Interpolation can create synthetic frames but does not make them independently observed content. A fixed-size tensor is a useful model interface, yet its shape does not establish that the intended temporal span or event ordering was preserved.

i(k)=arg min⁡i∣ti−τk∣,ek=ti(k)−τki(k)=\operatorname*{arg\,min}_i|t_i-\tau_k|,\qquad e_k=t_{i(k)}-\tau_k
This is one nearest-presentation-time rule with a separately specified tie-break and valid source times t_i. τ_k are target seconds and e_k signed selection errors in seconds. A causal latest-not-after rule is different; neither rule repairs missing source content by itself.↔ Scroll wide equations horizontally.

Use the sampling theorem only within its domain

For an ideal continuous signal bandlimited below a known highest frequency, exact uniform samples at a strictly higher rate than twice that frequency support the usual reconstruction argument with ideal assumptions. Real video involves finite exposure, occlusion, compression, camera motion, and potentially non-bandlimited changes. An anti-alias filter before downsampling can reduce frequencies outside the chosen band, while also removing information. It does not restore distinctions already aliased in the captured source.

SlowFast explores different temporal rates for complementary pathways, and TimeSformer studies space-time attention choices. Their task results motivate architecture and sampling experiments rather than a theorem that all fast motion is preserved. Report recognition quality by action frequency, duration, and source frame cadence where relevant. A task may tolerate lost high-frequency detail, or may depend exactly on it. Distinguish empirical task sufficiency from lossless reconstruction of an arbitrary physical scene.

cos⁡ ⁣(2π7n10)=cos⁡ ⁣(2π3n10),n∈Z\cos\!\left(2\pi\frac{7n}{10}\right)=\cos\!\left(2\pi\frac{3n}{10}\right),\qquad n\in\mathbb Z
This exact identity shows that uniform sampling at 10 samples/s cannot distinguish the stated 7 Hz and 3 Hz cosine signals. Frequencies are cycles per second and time samples n/10 seconds. It is a scalar temporal-signal counterexample, not a complete model of camera motion or video exposure.↔ Scroll wide equations horizontally.

Test both the sampling policy and the task

Build a timestamp fixture with irregular intervals and a synthetic signal with known frequencies. Test the selected times independently from the learned model so an indexing error cannot be explained away as model weakness. Then evaluate real task performance across rate and clip-span changes under the same split. If the model accepts timestamps as features, verify their units and handling of repeated frames; supplying an ordinal position still does not tell it the actual gap.

The waves lab provides a known analytic component that can be inspected at reachable times. Reset, set both phase difference 0 degrees and amplitude ratio 0, then predict the value at x=0 after four Steps: −sin(2)≈−0.9093. Reset and set both phase 0,ratio 1 for −2sin(2)≈−1.8186. Step spacing is half a second. This fixed-frequency analytic instrument does not downsample a video, run an anti-alias filter, or demonstrate the separate seven-versus-three-hertz example; it keeps the observed function and sampling times explicit.

Make two different temporal frequencies look identical

Sample x(t)=cos(2π·7t) and y(t)=cos(2π·3t) uniformly at 10 samples per second. Use exact ideal point samples, with t=n/10 and no noise or exposure averaging.

  1. At n=0 both values equal one. At n=1, cos(1.4π)=cos(0.6π)≈−0.309017.

  2. At n=2, cos(2.8π)=cos(1.2π)≈−0.809017. More generally, 7n/10=n−3n/10 and cosine is even and periodic, proving equality at every integer sample.

  3. The sampled sequences cannot identify which of these two source signals occurred. Increasing the model's capacity cannot remove that ambiguity without another assumption or additional observations.

The result

The different seven- and three-hertz signals share every ten-hertz point sample. This proves one information-loss mechanism under explicit sampling assumptions, not a universal motion-recovery claim.

Explore the model

Interactive lab

Interactive lab · waves

Name the analytic signal and the inspected timestamps

Reset → set both phase 0°,ratio 0; at x=0 predict −sin(2)≈−0.9093 after four Steps. Reset → set both phase 0°,ratio 1 for −2sin(2)≈−1.8186. The fixed wave has no video sampler, anti-alias filter, or adjustable seven-hertz component.

Name the analytic signal and the inspected timestampsTwo waves share the same frequency and wave number. Playback advances t in sin(2x − t) + a sin(2x − t + phase). Current readings: Sum at x = 0: 0.866; Resultant amplitude: 1.732; Phase difference: 60°. All parameters are adjustable with the labeled controls below.−3.14−1.5701.573.14−3.4−1.701.73.4x · radiansAmplitude
SumFirst componentSecond component
Sum at x = 0
0.866
Resultant amplitude
1.732
Phase difference
60°
t = 0 s

Two waves share the same frequency and wave number. Playback advances t in sin(2x − t) + a sin(2x − t + phase). Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forA temporal observation needs a known signal model and explicit times before supporting a reconstruction claim.

Check your understanding

An irregular-rate source is sampled by taking every fourth decoded image. Which claim is justified without further timestamp checks?

Try it yourself

Available source times are [0.94,0.98,1.07,1.11] s and the target is 1.05 s. Which frame does nearest-time selection choose, and which does a latest-not-after rule choose? Calculate each signed timestamp error.

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

Compare absolute distances for nearest selection, then exclude future presentation times for the second rule.

Compare with the solution

Nearest selection chooses 1.07 s, with error +0.02 s. Latest-not-after chooses 0.98 s, with error −0.07 s. The first is closer in presentation time but uses later content than the target; the second obeys that presentation cutoff while having larger temporal error. A live task additionally needs decode-ready availability, so even the earlier PTS is not by itself proof it was usable at the decision instant.

Take it with you

  • Frame strides and timestamp grids coincide only under established cadence assumptions.
  • Aliasing creates indistinguishable observations, and reconstruction claims need explicit signal conditions.
  • Record selected timestamps, repeats, gaps, and task-specific quality when changing sampling.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.