Tensor strides: prove which element an address represents

Let strides count storage elements rather than bytes, and let the storage offset locate the logical origin within a backing allocation. Multiply the resulting element offset by bytes per element before adding it to the storage base. Each logical index must be within its dimension, and the complete typed access must stay within valid storage. Checked arithmetic, alignment, initialization, lifetime, and permitted aliasing remain separate obligations. Merely multiplying dimensions gives a logical count; it does not establish the highest accessed address of a padded view.

A transpose can exchange dimensions and strides without moving any element. The new logical element at column-row coordinates then refers to the same original storage as the corresponding row-column element. A slice can change the origin and introduce gaps. Some APIs support only particular stride classes; do not assume that a generic signed-stride formula means every framework accepts negative strides. PyTorch's documented view operations share underlying data, while reshape and contiguous conversion have operation-specific conditions determining whether storage must be copied.

For the usual dense row-major convention with every dimension larger than one, the last stride is one and each earlier stride is the product of later dimension sizes. Singleton dimensions can allow additional equivalent stride values because their index never advances. Frameworks also recognize alternative memory formats, such as channel-oriented layouts. State which definition a kernel dispatch checks. A shape alone cannot prove row-major contiguity, and an array occupying one backing allocation can still contain gaps between logical rows.

Broadcast views can use a zero stride so many logical coordinates read the same stored value. That is useful for a read-only bias but dangerous for an output kernel that assumes each coordinate owns an independent destination. Overlapping views create related aliasing obligations even when strides are nonzero. A correct general-stride reader is not automatically a correct general-stride writer. Either reject unsupported overlap, use a semantics-preserving implementation, or establish an independent-output contract before parallel writes begin.

A contiguous conversion can simplify downstream indexing and improve some access patterns, but it introduces a copy and additional live storage. Include that conversion in an end-to-end benchmark unless the production pipeline genuinely reuses the converted tensor enough to amortize it. Check whether the consuming operation already handles the given strides efficiently. For GPU code, logical stride affects which addresses adjacent lanes request, while coalescing and caches depend on the actual architecture and access grouping. Avoid declaring every larger stride slow without tracing addresses.

The cache lab illustrates only a deterministic address cycle. Reset, set C=4 and s=1, predict zero hits among its sixteen initial requests, then Play or Step. Separately Reset, set C=4 and s=4, predict twelve hits, then Play or Step. The larger toy stride visits only four residues modulo sixteen, so this comparison is not a rule that larger tensor strides improve GPU access. It contains no tensor metadata or memory transactions. Use it to separate a measured working set from a layout label, and do the actual tensor-address proof on paper.