Microbenchmarks: measure the question you actually asked
Design controlled latency and throughput experiments while distinguishing core cycles, elapsed time, and timestamp-counter ticks.
On this page
The question for today
What you’ll understand
- Design different instruction sequences for latency and throughput questions.
- Explain why timestamp readings need documented ordering and counter units.
- Report a benchmark with enough context to reproduce and interpret it.
There is no single instruction cost that predicts every program. A dependent chain asks how soon one result can feed the next operation. Independent operations ask how rapidly available resources can accept work. A memory-dependent kernel asks still more questions about addresses, cache state, and translation. A useful microbenchmark isolates a defined question while reporting the assumptions that make the measurement interpretable.
Construct the dependency structure intentionally
To estimate producer-to-consumer latency, create a chain in which every operation needs the previous result. Ensure the final answer is observed so the compiler cannot discard the computation, and inspect the generated instructions to confirm that the intended chain exists. Loop bookkeeping and measurement overhead must be separated or amortized. Some instruction forms have different input-to-output dependency paths, so state operand types and forms instead of attaching one unexplained number to a mnemonic.
To explore reciprocal throughput, provide enough independent chains that latency does not force every issue slot to wait. Increase their count cautiously: too few leave dependencies exposed, while too many can create register spills or different front-end pressure. The best isolated throughput does not automatically predict a full workload with loads, stores, branches, and other operations sharing resources. A microbenchmark answers its constructed scenario; the application comparison decides whether that scenario was relevant to users.
A timer has semantics too
RDTSC reads an x86 timestamp counter but is not a serializing instruction. Without appropriate documented ordering, surrounding work may not line up with the apparent measurement boundaries. The exact sequence of fences, timestamp instructions, and compiler barriers must match the experiment and processor guidance. Do not present an unqualified timer snippet as a universal solution. Operating-system scheduling, interrupts, migration, power management, and virtualization can also affect the result or the interpretation of the counter.
An invariant timestamp counter advances at a stable reference rate independent of changing core frequency; its ticks are therefore not necessarily actual core cycles. Conversely, a core-cycle performance event counts according to its own documented scope and conditions. Elapsed seconds, TSC ticks, and core cycles can all be useful if named correctly. Record which one was collected, how it was converted, and whether the process could migrate across dissimilar cores. Avoid turning a rate assumption into an architectural guarantee.
Use repeated evidence and test a relevant prediction
Warm up the chosen code and data state when steady-state behavior is the target; measure cold behavior separately when startup matters. Repeat runs, report variability, and distinguish the minimum observed interference from a representative distribution. Pinning a thread can control one source of variation but does not freeze all power and memory behavior. Record processor model, relevant features, compiler version and flags, code bytes or assembly, data sizes, and loop structure so another engineer can assess the measurement.
The binomial lab can illustrate the uncertainty of a hypothetical pass/fail event across independent trials: choose an assumed p and n and predict the count distribution. It does not collect timing samples or fit latency distributions. Real timing observations can be correlated and continuous, so this lab cannot validate benchmark significance. For a real comparison, define the expected improvement first, examine timing distributions, and then run the application workload. If the isolated win disappears, update the performance explanation rather than hiding the broader evidence.
Work it through
Do not label reference ticks as core cycles
A controlled run takes 24 million ticks on a verified 2.4 GHz invariant timestamp counter and performs six million tested operations. A separate matching core-cycle measurement reports 30 million cycles.
Convert the timestamp delta using its reference rate: 24 million divided by 2.4 billion is 0.01 seconds, or 10 milliseconds. This calculation reports elapsed time under the documented measurement boundaries.
Divide operations by elapsed time to obtain 600 million operations per second. Separately, the core-cycle event gives 30 million/6 million=5 core cycles per operation for that tested sequence.
The timestamp ratio is four TSC ticks per operation, not four actual core cycles. The two counters can differ because their rates and counting definitions differ. Neither average alone identifies an isolated instruction’s dependency latency.
Report 10 ms, 600 million operations/s, four reference ticks/op, and five measured core cycles/op with their distinct scopes.
The living notebook
Explore the model
Interactive lab · probability
Trial count under an explicit assumption
Set p and n to inspect an exact binomial distribution for independent success events. Predict the mean and variance before changing n. The lab does not sample timings, benchmark instructions, or establish that real measurements are independent.
- Mean · np
- 5
- Variance · np(1−p)
- 2.5
- Total probability
- 1
These are exact binomial probabilities for independent trials with one fixed success probability. Playback tours the distribution; it does not resample outcomes. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.
Look forAn experiment needs a stated observation model; a convenient diagram cannot supply unmeasured assumptions.
Pause & reason
Check your understanding
Your turn
Try it yourself
A dependent-chain test measures 9,000 core cycles for 2,000 operations. A comparable empty harness costs 1,000 cycles. Estimate latency and name a required validation.
Take a moment to make a prediction or work it out on paper. Open a hint when you need one.
Give me a hint
Subtract only comparable overhead and keep the dependency-chain assumption explicit.
Compare with the solution
The estimate is (9,000−1,000)/2,000=4 core cycles per operation. Inspect assembly to verify the intended dependency and operation forms, ensure comparable overhead, and repeat controlled runs. An average from unrelated independent operations would instead address a throughput scenario.
Keep the idea
Take it with you
- Latency chains and independent throughput tests are distinct experiments.
- Counter ticks require documented units and ordering.
- Use reproducible microbenchmarks to form explanations, then validate application impact.
Read further
Original sources and deeper explanations to follow at your own pace.
Optional progress, saved in this browser when available. Revisit any lesson at any time.
Lesson not marked complete.