Hardware counters as evidence

Determine whether an event counts retired operations, speculative activity, requests, fills, cycles, or something else. A label containing 'cache miss' does not necessarily mean all misses at every level or exactly one transfer from DRAM. Event encodings and support vary with processor model and sometimes with core type. Use the target's documentation and the tool's resolved event description, then record the exact configuration so another person can reproduce the same quantity.

Scope also matters. A process measurement, a per-thread measurement, and a system-wide count can include different work. User-only filters exclude kernel execution that may still contribute to elapsed time. Thread migration or hybrid cores may complicate which events were available. Wall time includes waiting and descheduling, while counted CPU cycles describe selected execution contexts. A low cycle count during a slow request can indicate waiting elsewhere rather than a remarkably efficient computation.

A processor exposes a limited number of programmable counters. Requesting more simultaneous events may cause the measurement system to time-share them. A common scaling procedure multiplies the raw count by enabled time divided by actual running time. This estimates the full-interval count only when the observed portions represent the unobserved portions well enough. Report running coverage; a large scaled number with tiny coverage can convey unwarranted precision.

Phase changes are especially dangerous. If one event runs mostly during initialization and another mostly during steady computation, their individually scaled totals can produce a misleading ratio. Grouping compatible events can help obtain simultaneous observations, but hardware constraints may prevent scheduling a group. A measurement that reports an event as unsupported or not counted is missing evidence, not a zero. Reduce the event set and repeat a representative workload before drawing conclusions from the ratio.

Suppose a layout change reduces both elapsed time and one miss metric. That supports a locality hypothesis, but a simultaneous reduction in instructions or a changed thread count may provide another explanation. Hold useful work fixed, preserve correctness, and compare repeated runs with warmup and initialization treated consistently. Normalize counts by meaningful work units such as processed records when that makes cross-run comparisons clearer. Retain absolute totals because a ratio can improve while total cost grows.

The probability lab shows exact binomial distributions for independent trials with common probability p. Set p=0.5 and compare n=10 with n=30: predict a larger absolute standard deviation but a smaller standard deviation of the success fraction. This is a statistical contrast, not a counter-error estimator. Real measurements may contain correlated phases and systematic bias; additional repetitions do not remove those automatically. The lab cannot infer confidence intervals for multiplexed hardware counters from one running-percentage value.