Tensor trains and matrix product representations

For a real d-way tensor, core Gₖ has dimensions rₖ₋₁ by nₖ by rₖ, with boundary ranks r₀ = r_d = 1. Fixing the physical index iₖ leaves a matrix of size rₖ₋₁ by rₖ. Multiplying these matrices in chain order produces one scalar entry of the tensor. Expanding that multiplication sums all internal bond indices, so each bond index appears in two neighboring cores.

A representation stores the sum of rₖ₋₁nₖrₖ entries across its cores. If every physical dimension is at most n and every internal rank at most r, this is O(dnr²), with smaller boundary terms. The estimate assumes ranks stay bounded; generic tensors can require exponentially large central ranks. Also, efficient storage of factors does not permit materializing all dense entries without paying for the output size.

At a cut after mode k, group the first k indices into a row index and the remaining indices into a column index. Within each group use the first listed index as the fastest. Contracting the left and right portions of the train gives a matrix factorization through the bond index, so this unfolding rank is at most rₖ. Conversely, successive exact matrix factorizations construct a train whose bonds equal these separation ranks.

These prefix–suffix ranks differ from the individual mode ranks used by Tucker. Their values depend on the ordering. For the four-mode tensor δᵢ₁ᵢ₃δᵢ₂ᵢ₄ with each index taking two values, the cut (1, 2)|(3, 4) gives a four-by-four identity matrix and rank four. Reordering to (1, 3)|(2, 4) turns that central view into an outer product of two vectorized identities, with rank one. The surrounding cuts still have rank two. The data are unchanged up to index permutation, but the required chain structure improves.

Insert an invertible matrix S on one side of a bond and its inverse on the other. The neighboring factors change, but their product and every tensor entry remain identical. This gauge freedom explains why core entries themselves are not uniquely determined by a tensor. Orthogonalization chooses convenient gauges so that contractions and local least-squares problems have controlled numerical behavior.

Truncating singular values in a sequence of orthogonalized cuts gives TT-SVD or train rounding. The accumulated Frobenius error is governed by squared discarded tails, not by adding arbitrary core-entry errors. Approximation can change the needed ranks significantly when singular values decay. Before selecting a bond budget, inspect the relevant cut spectra and decide which absolute or relative error target the application actually needs; a single rank value does not express both storage and accuracy.