Tucker decomposition and the higher-order SVD

For a real d-way tensor, choose matrices Uₙ with orthonormal columns spanning selected mode subspaces. Contract T with each transposed factor to form the core G. Expanding that core through the original factors gives the orthogonal projection of T onto the product of the selected subspaces. If every mode subspace contains the corresponding fiber span, reconstruction is exact. The mode ranks of the reconstructed tensor are bounded by the numbers of columns retained.

Orthogonality makes this projection easy to interpret in Frobenius norm. Expanding a core through orthonormal factors preserves its norm, and the residual is perpendicular to the represented product subspace. Nonorthogonal factors are also valid Tucker representations, but a core norm then no longer equals the norm of its expansion automatically. This is why numerical algorithms often orthogonalize the factors before reporting compression errors.

The HOSVD takes left singular vectors from every mode unfolding. With complete orthogonal factor matrices it is an exact basis change, and the core has orthogonal mode slices with their norms ordered by the corresponding unfolding singular values. This is called all-orthogonality. It does not mean that every off-diagonal core entry vanishes, and the tensor is not generally reconstructed by matching just one component index across all factors.

Truncated HOSVD retains leading singular vectors in each mode and projects onto their product space. It has a controlled approximation guarantee, developed in the next lesson, but generally does not solve the best fixed-multilinear-rank optimization exactly. Repeated singular values also allow rotations inside a tied singular subspace. Different factor matrices can therefore describe the same projected tensor or equally valid decompositions without representing a failure of the SVD computation.

A Tucker representation with rank bounds r₁ through r_d stores a core with product rₙ entries plus factor matrices with a total sum nₙrₙ entries. This raw count includes basis freedoms; it is not the dimension of a space of distinct tensors. Rotating one factor and applying the inverse rotation to the matching core mode preserves the represented array. Gauge choices can improve conditioning without changing data.

Compression is useful only if this count and the costs of later operations improve on dense storage. A small tensor can have a larger Tucker description than its original array. For example, full factors and a full core merely add overhead. In high order, the core product may itself become enormous even when every mode rank is moderate, motivating tensor trains and other network formats instead of assuming Tucker alone removes dimensional growth.