Map local work and global traffic separately

A two-dimensional mesh has many short links, but a balanced left-right cut intersects only a column of links. Adding processing elements increases local arithmetic capacity faster than some global communication cuts. For a given placement, count bytes that must cross each relevant cut and compare them with its directional capacity. Opposite directions may use distinct full-duplex resources; do not add both capacities when all required bytes flow one way. Routing and competing traffic can reduce achievable rate below the cut bound.

Nearest-neighbor stencils can reuse local links well, while transposes and broad reductions can move data through many hops. Hop count contributes contention and buffering demand even if each message’s endpoint distance is modest. Treat a communication collective as a scheduled algorithm rather than an abstract operation with zero cost. The wafer-scale FFT and Reduce papers study specific communication structures and implementations; their results should be read with their problem sizes, precision, and machine generation attached.

Local memory capacity determines how much of each tensor shard and its temporary state can remain near computation. Include live inputs, outputs, partial sums, and routing buffers rather than only weights. Overly fine partitioning can reduce per-element storage while increasing halo or reduction traffic. Overly coarse partitioning can exceed local capacity or leave too little parallelism. Search the interaction between shard shape, compute granularity, and communication pattern, then validate the chosen mapping.

Within-wafer bandwidth is not interchangeable with off-wafer ingress, egress, or external weight supply. A model that exceeds local capacity may need another execution strategy, additional storage, or explicit streaming. Include that traffic and its scheduling in end-to-end time. Likewise, scaling across several wafer-scale systems introduces another interconnect tier. A measured local-kernel result does not establish that a large training job, checkpoint, or input pipeline scales at the same rate.

Explain the numerical representation, accumulation, reduction order, and any precision changes used by the mapping. A high-throughput local operation is meaningful only if the overall algorithm meets the quality or tolerance target. Collective ordering can change floating-point results, so validate more than a single output bit pattern unless bitwise equivalence is actually required. Retain problem size and placement with every timing to prevent favorable small cases from standing in for capacity-limited ones.

Measure strong and weak scaling separately, include external data loading when it is part of the application boundary, and report mapping or compilation costs when they matter operationally. Monitor hot links, local-memory pressure, and idle processing elements to test the cost model’s explanation. Vendor peak arithmetic and aggregate on-wafer link totals describe capability ceilings, while measured workload time depends on how that workload uses them. Neither a large wafer nor many cores supplies a universal advantage for every communication graph.