Differential testing needs a valid reference domain

Record shapes, dtypes, layouts, allowed aliasing, integer overflow rules, floating-point mode, and operation-specific preconditions. Division by zero or out-of-range conversions may be undefined or produce poison under the selected IR contract rather than supply a meaningful reference output. LLVM 20.1.0 documents those distinctions explicitly. Exclude invalid test inputs or test the appropriate defined failure behavior; do not infer a compiler bug merely because two invalid executions happen to disagree.

For strict operations, compare required exact behavior, including signed zero or exceptional values where the contract preserves them. For an approximate or reassociated computation, use a tolerance justified by its actual specification and workload, with NaN and infinity handling defined separately. A model-level quality threshold is not automatically a valid per-operation compiler tolerance. Keep numerical-mode changes visible so a strict baseline is not silently replaced by a more permissive reference just to make tests pass.

Use zero and small dimensions where valid, values around tile and vector boundaries, non-square shapes, noncontiguous views, nonzero offsets, and permitted overlap. Include cancellation and conversion boundaries for floating-point transformations. A random dense tensor often misses a one-element tail or an old-value consumer. Memory checks and guarded allocations complement value comparisons because an out-of-bounds read can leave outputs apparently correct while violating the execution contract.

Compare intermediate representations or stage outputs when possible: tensor semantics, post-bufferization behavior, scheduled loops, vectorized code, and the final kernel. A failure appearing after one pass narrows the investigation but does not automatically prove that pass is at fault; earlier invalid IR may only become observable later. Preserve verifier diagnostics, compiler revision, flags, target, and a minimized reproducer. Reduction should retain the legality assumptions and numerical mode that made the original case meaningful.

A reference that shares the same faulty indexing helper can reproduce the same bug and falsely agree. Use an independently understandable small implementation and manually checked cases alongside randomized testing. For pure independent elementwise operations, a proven permutation relation can provide another check; do not invent algebraic metamorphic rules that fail under floating-point or side effects. Cross-target differences can reveal issues, but compare their supported semantics and allowed precision before declaring one target wrong.

The function lab illustrates a known analytic reference only. Reset and set both coefficient a=1 and position x=1, predict y=1, tangent slope 2, and integral from 0 to 1 equal 1/3, then inspect with Step. It computes y=ax² and associated formulas, not a tensor compiler or test generator. An implementation matching this small family does not validate arbitrary lowering. Use the contrast to explain why a few passing inputs are useful evidence yet cannot establish the universal correctness statement above.