Vector execution: autoreset modes change which rows are transitions

In the pinned SAME_STEP mode, an ending environment first produces its final transition, then immediately resets. The returned ordinary observation for that slot is the new initial observation, while final_obs and final_info carry the old ending transition’s data in the information dictionary. Vector info entries have associated presence masks, including _final_obs, so a collector must check whether a field exists for each environment rather than interpret an unfilled object-array entry as a valid observation. Rewards and ending flags still describe the action that ended the old episode.

In NEXT_STEP mode, the ending step returns its actual final observation. On the following vector call, that slot resets instead of executing the supplied action and returns zero reward with false ending flags. That reset-only output must not become a fabricated zero-reward transition from the old final state. Other vector slots may execute real actions during the same call. DISABLED mode instead requires correctly resetting ended environments through the supported interface before stepping them again. These distinctions affect sample counts, hidden-state handling, and bootstrap inputs.

At an externally truncated transition, a continuing-task value can bootstrap from the old final observation. The subsequent reset episode must still begin a new sequence with its appropriate recurrent state. A value mask that retains truncation bootstrapping therefore cannot also serve blindly as the recurrent or trace-continuation mask. At a collector-only fragment boundary, similarly retain a suitable tail value while stopping the local backward recursion. Preserve which boundary was encountered so that a sequence assembler cannot join unrelated episodes simply because their numeric environment-slot indices match.

A recurrent critic may need the old trajectory’s hidden context to value the final observation, while the next episode needs reset state. Evaluate or store those contexts before overwriting them. A collector that resets every hidden state whenever any slot ends also corrupts the continuing environments in the batch. Test mixed batches containing ordinary, terminated, truncated, and reset-only slots, with clearly tagged episode identifiers. Correct vectorization is a per-slot state machine, not a global Boolean reset applied to the entire batch.

Vectorization can amortize policy inference while introducing a slowest-slot barrier, reset costs, serialization, or interprocess overhead. A serial SyncVectorEnv does not parallelize simulator execution merely because observations are batched. Async execution can overlap independent work but still needs correct action-to-observation matching and bounded memory. Count valid action-executing transitions and completed episodes separately from vector calls. Otherwise a next-step autoreset configuration can appear to collect more training data simply by counting reset-only outputs as experiences.

Reset the probability lab, explicitly set p=0.75 and n=20, predict the mean valid-slot count in a hypothetical independent binary model, then Play or Step. Compare a separate Reset with p=0.5 and n=20 explicitly set. The means are fifteen and ten. Real vector reset events depend on episode lengths and are generally not independent identical trials. Playback highlights exact binomial outcomes, not actual environments or sampled resets; this toy model only distinguishes total slots from assumed useful-transition counts.