Measure the boundary instead of declaring every graph break a failure

In PyTorch 2.9's documented partial-capture model, unsupported operations or data-dependent host behavior can lead to eager execution between captured regions. A value inspection that requires a device result on the host can also expose synchronization cost, though a graph break does not universally imply a device synchronization. Capture options and supported operations are version-sensitive. Inspect diagnostics and a profiler for the exact program instead of inferring cost solely from the number of graphs shown in a report.

Some boundaries are intentional. A request logger might be tiny relative to the matrix product before and after it, or an infrequently executed error path may not justify aggressive capture work. Conversely, a break inside a short operation repeated hundreds of times can dominate launch and dispatch costs. Count frequency as well as duration. Preserve equivalent input preparation, host behavior, and device completion when comparing alternatives; otherwise removing a log operation or failing to wait for work can look like a compiler speedup.

Combining regions may avoid materializing an intermediate or launching another kernel. It can also enlarge live ranges, increase register pressure, alter scheduling opportunities, or create a more expensive compilation. These effects depend on operation shapes and the backend. A fully captured graph is therefore useful evidence about representability, not a universal certificate of faster execution. Choose performance hypotheses with an identified mechanism, such as removing a repeated host round trip, then test that mechanism with controlled inputs.

For memory-sensitive operations, reduced traffic can raise a throughput ceiling without establishing the achieved rate. The roofline model divides operation count by bytes moved at a stated memory boundary; its bandwidth term is an upper bound under that model. Do not substitute allocation size for actual traffic or count a read only once when the implementation reloads it. Fusion can change both traffic and other bottlenecks, and a boundary change may affect dispatch without changing arithmetic intensity at all.

Report the initial call and steady compatible calls separately, then add a workload that changes relevant guards. Include repetitions and a declared aggregation rule rather than selecting the fastest sample without context. For asynchronous accelerators, wait for the result at the chosen measurement boundary, as the JAX JIT documentation demonstrates. Also record what warm-up did: it may populate artifact caches, memory pools, and device libraries. Those effects are real parts of the experiment, not automatically costs of the same compiler stage.

In the roofline lab, Reset and set both I=8 FLOP/byte and B=80 GB/s. Predict the ceiling 640 GFLOP/s, then Play or Step to trace it. Reset and set both I=16,B=80 for the separate ceiling 1,000 GFLOP/s. These are hypothetical intensity choices rather than the measured effect of deleting a graph break. Playback only traces the selected roof. Use this instrument to distinguish a potential traffic benefit from a guaranteed runtime saving, then return to the actual program's synchronized measurements.