Kernel harnesses: verify the operation before timing its boundary

Start with small hand-computable cases and a trusted implementation using the same input values, axes, masks, output layout, and precision policy. A higher-precision reference can evaluate represented low-precision inputs more accurately, but it does not restore values discarded before the inputs reached the kernel. Check a range of absolute and relative errors rather than comparing only an average. Cancellation, near-zero references, softmax tails, constant normalization rows, and overflow boundaries need deliberate cases. Nonfinite outputs require explicit handling rather than relying on a comparison that silently treats NaN as acceptable.

Exercise dimensions just below and above tile boundaries, noncontiguous supported views, singleton and zero-size cases under the API contract, and all-masked rows where relevant. Verify unsupported overlap is rejected or handled by its defined path. Memory tools such as NVIDIA Compute Sanitizer can help identify invalid accesses, races, or synchronization mistakes within their documented coverage. Passing numerical comparisons cannot prove that no out-of-range write occurred elsewhere, while a clean memory check does not establish the intended formula. Use both kinds of evidence for the kernels that need them.

Warmup can separate context initialization, compilation, autotuning, allocator setup, and changing device state from steady execution. Include those costs in a cold-start measurement if the product actually pays them; exclude them only from a clearly labeled steady-state boundary. GPU launches are asynchronous, so a host timer around a launch can measure submission rather than completion. CUDA events recorded around the intended work in the appropriate stream can measure a device timeline interval when completion is properly awaited. Other streams, synchronization, and concurrent workloads must remain part of the stated experiment.

Reuse allocations or capture launch sequences only when that reflects the boundary being compared. A captured graph and an ordinary launch path answer different overhead questions unless both are relevant production choices. Use enough repetitions for timer resolution and variability, but avoid reporting a batch average as a per-call latency percentile. Record device model, software versions, compiler flags, precision modes, shapes, strides, and random seeds. Profiling and sanitizer runs can perturb timing, so collect mechanism evidence and representative performance measurements with their instrumentation conditions clearly distinguished.

Compare a tuned library or prior kernel with matched semantics and a representative shape portfolio, including small, thin, odd-sized, and large cases. Report failures and unsupported cases rather than silently omitting them. Check whether the claimed bottleneck is reflected in bytes, instructions, occupancy constraints, stalls, or launch overhead. A roofline ceiling can guide investigation, but a counter using a different byte boundary cannot prove the ceiling was violated. End-to-end model throughput may depend on data preparation, layout conversion, and other kernels that an isolated test excludes.

The roofline lab offers a unit-consistent synthetic reference only. Reset, set I=5 and B=100, predict ceiling 500 GFLOP/s, then Play or Step. Separately Reset, set I=10 and B=100, predict the fixed 1,000 GFLOP/s ceiling, then Play or Step. It does not run a GPU, compile code, measure bytes, or generate benchmark samples. Use its decimal units to check a report's dimensional arithmetic, then require actual synchronized timings and correctness results for a real performance claim.