Treat symbolic dimensions as a domain with obligations
Propagate two-dimensional tensor shapes, preserve element counts, and distinguish mathematically valid constraints from what a compiler can establish.
On this page
The question for today
What you’ll understand
- Derive output and reshape dimensions from explicit input domains.
- Identify divisibility and range constraints required by an unmasked tiled path.
- Explain why an inconclusive symbolic analysis is not a mathematical counterexample.
A symbolic dimension is not permission to accept every imaginable array. It denotes values in a declared domain, and each operation adds obligations. A matrix product requires matching contraction dimensions; a reshape preserves element count; an unmasked tiled path may require divisibility. These conditions should be derived from the program and checked at the appropriate boundary. They should not be invented after a convenient example happens to work. A useful analysis records both the mathematical requirement and the compiler mechanism that establishes or enforces it.
Derive shapes using shared symbols rather than coincidences
Let X have shape [B,4] and W shape [4,3], with integer B between one and sixty-four. The product has shape [B,3] because the two contraction dimensions equal four. If a bias has shape [3], row broadcasting can preserve that output shape under the chosen operator semantics. Giving two unrelated input axes the same symbol asserts a relationship; it is stronger than seeing both equal eight in one example. Independent symbols should remain independent unless the contract or a checked condition links them.
Shape inference should also track the domain in which expressions are meaningful. Dividing a dimension by two requires more than writing a fraction in a type annotation when the target dimension must be an integer. Zero-length dimensions, symbolic rank, and value-dependent output sizes have different support across systems and operations. This lesson deliberately uses positive batch sizes and fixed rank. If a production API accepts empty batches, test and represent them explicitly rather than quietly importing this narrower domain into its interface.
Preserve counts and distinguish constraints from implementations
A contiguous X of shape [B,4] can be logically reshaped to [B,2,2] because both contain four B elements. By contrast, grouping the rows into blocks of four to obtain [B/4,4,4] requires B divisible by four. If B is six, silently truncating B/4 discards a contract obligation; it does not solve the reshape. A general implementation could use a different representation or handle a remainder, but that is a different program from the unmasked full-block path.
PyTorch 2.9 export and dynamic-shape documentation describe recorded shape conditions associated with a captured program. JAX's shape-polymorphism documentation also explains that its symbolic reasoning is intentionally limited. Failure to prove an inequality can therefore mean analysis is inconclusive, even when a human can establish it mathematically. Conversely, a sample input satisfying a condition does not prove every declared input satisfies it. Keep the proof, runtime check, rejected input, and unsupported analysis outcome as distinct cases.
Test the edges of the domain
Construct shape tests around each obligation: the minimum and maximum batch, a valid multiple of the tile size, and a nearby invalid remainder. Check shared-symbol relationships by deliberately varying only one input dimension. The expected response outside a specialized domain might be rejection, a guarded alternate artifact, or a remainder-capable path. It must be specified rather than inferred from a successful in-domain run. An unchecked assumption is particularly dangerous when it controls memory indexing instead of merely changing a diagnostic message.
The modular lab isolates the remainder arithmetic used by one divisibility condition. Reset, set both modulus m=4 and additive step a=3, predict residues for candidate batches B=3n, then Step four times. Each half-second Step advances one addition, giving residues three, two, one, zero. Reset and set both m=4,a=4 to compare batches divisible by four at every positive step. This bounded integer orbit does not propagate tensor shapes, prove range constraints, or validate a real exported artifact.
Work it through
Check a blocked batch reshape
A fictional tensor X has shape [12,4]. A full-block path reshapes it to [B/4,4,4] for positive B divisible by four. A later request has shape [10,4]. No padding or remainder path is implemented.
For B=12, the input contains 12×4=48 elements. Since 12/4=3 is an integer, [3,4,4] also contains 48 and satisfies the declared grouping.
For B=10, the input contains forty elements. Using floor(10/4)=2 would produce [2,4,4] with thirty-two elements, losing eight; the desired exact leading dimension 2.5 is not an integer tensor dimension.
The second request must not execute this specialized reshape unchecked. Reject it or select a separately correct implementation according to the interface contract; a successful twelve-row example is insufficient evidence.
Twelve rows satisfy the blocked domain and ten do not. The failure is a specific integer-count obligation, not a general inability to represent dynamic batches.
The living notebook
Explore the model
Interactive lab · modular
Predict a divisibility guard's remainder arithmetic
Reset → set both m=4,a=3; predict the remainders of B=3n, then take four Steps to obtain 3,2,1,0. Reset → set both m=4,a=4 for an all-zero remainder comparison. The additive orbit is not a symbolic-shape solver or compiler guard executor.
- Residue r0
- 0
- Orbit period · m/gcd(m,a)
- 12
- Distinct residues visited
- 1 of 12
Addition modulo m starts at zero: rₙ = na mod m. Playback advances two additions per second. The orbit has period m/gcd(m,a), including period one for a = 0. This is not a cryptographic security test or a primality algorithm. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.
Look forA valid full-tile domain needs an established divisibility condition.
Pause & reason
Check your understanding
Your turn
Try it yourself
For X shaped [B,4] with 1≤B≤64, a path groups rows in eights with no remainder handling. State the valid B values and check B=24 and B=26. What extra input-domain requirement has the optimization introduced?
Take a moment to make a prediction or work it out on paper. Open a hint when you need one.
Give me a hint
Require a positive integer q with B=8q, and intersect that set with the original batch interval.
Compare with the solution
The valid batches are 8,16,24,32,40,48,56,64. Twenty-four equals 8×3 and can form three complete row groups; twenty-six leaves a remainder of two. The specialized path has introduced B divisible by eight in addition to the original range. That condition must be guaranteed or checked before dispatch, with the specified rejection or alternate behavior for other batches. It cannot be justified by the original interval alone.
Keep the idea
Take it with you
- Symbolic dimensions carry integer, range, and relationship constraints.
- Reshape and full-tile optimizations create explicit count and divisibility obligations.
- An analysis limitation is distinct from a false constraint or an accepted unchecked input.
Read further
Original sources and deeper explanations to follow at your own pace.
Optional progress, saved in this browser when available. Revisit any lesson at any time.
Lesson not marked complete.