Count source identities and sampler multiplicities across workers

A stable clip record can include a source-content/version identifier, selected stream, source timestamp interval, and sampling-policy version. A separate view identifier records augmentation parameters or their deterministic derivation. Two views of one clip are not two independent recordings, and two overlapping intervals are not necessarily independent examples. Assign source groups to splits before generating clips so adjacent windows, re-encodes, and crops cannot cross a prohibited group boundary merely by receiving different filenames.

An iterable loader must partition work across both distributed ranks and local worker processes. PyTorch 2.9 documents that each worker gets its own dataset copy, so an unsharded iterator can replay the same records in every worker. One simple deterministic assignment uses a global worker identifier when every rank has the same worker count. More elaborate load balancing is possible, but it must preserve the declared inclusion probabilities and identities rather than hiding repeats behind faster workers.

For a fixed-size dataset, PyTorch 2.9 DistributedSampler with drop_last=False adds indices so ranks receive equal-length lists; with drop_last=True it drops a tail to divide evenly. Its shuffle order also requires the documented set_epoch call before creating each epoch iterator. Every rank must use compatible dataset ordering, seed, and epoch settings. The sampler's tail policy differs from the DataLoader's batching drop_last option, which acts on batches and should be recorded independently.

Repeated indices may be intentional for training, but evaluation needs its own policy. If repeated occurrences of a clip have identical losses, known multiplicities allow a unique-clip average through inverse-multiplicity weighting. Deduplicating by stable clip ID is another option when predictions are identical and the task permits it. Stochastic views can produce different losses, so this algebra does not prove their gradients or metrics equal a single canonical view. Decide whether the estimand is over clips, sources, or augmented draws.

Emit an audit trace containing source/clip/view IDs, rank, worker, epoch, and whether an occurrence was introduced by padding. Compare its multiset with the intended manifest, checking missing records and multiplicities rather than only total counts. Include a tiny dataset smaller than or not divisible by the rank count. A resume test must specify whether it restores iterator position exactly or restarts an epoch with an accepted repeat policy; resetting the random seed alone may not restore prefetched worker state.

The modular lab makes ownership arithmetic visible. Reset, set both m=3,a=1, predict residues 0,1,2,0, then take three Steps. Reset and set both m=6,a=1 to represent six distinct worker slots and predict a six-step cycle. Edits retain elapsed time, so compare fresh runs. This counter has no DataLoader, distributed sampler padding, video costs, augmentation randomness, or recovery cursor. Use an actual ID trace to establish coverage and a measured cost trace to assess balance.