Correlate events before subtracting timestamps from different clocks

A host interval around an asynchronous API call often measures enqueue and runtime overhead rather than the device operation's finish. A device event records progress in the stream and context covered by its contract. To time a region, place events so required work lies between them and wait for completion before reading the elapsed result. Independent-stream work may fall outside that interval unless explicit dependencies bring it inside. Synchronizing after every operator changes execution and can destroy the overlap the experiment was supposed to measure.

Keep CPU thread, process, device, stream, rank, and step identity with each event. A correlation identifier can connect a host API call with resulting device work even when their durations differ greatly. A range annotation describes a chosen logical region; it is not automatically a device synchronization boundary. Tool support varies by version and collection mode, so inspect the captured metadata rather than assuming every operation has a complete cross-layer link. Missing correlation can also follow a capture window that begins after an API call but includes the resulting execution.

Two hosts can have different offsets and slightly different clock rates. Converting units alone removes neither effect. A collector may estimate offsets, correlate clocks, or use an external synchronization service; each method has residual error that can change with network conditions and elapsed time. If two mapped events are only a few microseconds apart while their joint uncertainty is larger, their timestamps cannot establish a strict order. State the interval of possible differences instead of reporting the midpoint as exact causal evidence.

Dependencies can establish order even when clock uncertainty prevents a timestamp proof. For example, a consumer that waits on a correctly identified producer event cannot consume that result before the producer's required completion. Conversely, merely observing one event earlier than another does not prove that it caused the second. Combine clock alignment with stream ordering, data dependencies, event waits, and matching collective identities. Do not subtract unrelated devices' raw event timestamps as if one shared stopwatch generated them; use the timing and correlation mechanisms the tool actually supports.

Record tool and driver versions, clock-correlation settings, capture start and end, warmup, and which ranks were observed. Check that the region includes both correlation-producing calls and their resulting activities. Compare an uninstrumented completed interval with the instrumented run to expose added waits. For device-event measurements, document stream placement and any synchronization used only at the region boundary. For cross-host analysis, preserve the measured correlation error; a clean-looking joined timeline does not eliminate uncertainty or guarantee that a short apparent gap is real.

The wave lab offers a simple phase comparison rather than a clock-calibration tool. Reset, set both phase difference 0° and amplitude ratio 1, predict sum zero at x=0 and time zero, then Step once: after 0.5 seconds the sum is −2 sin(0.5), about −0.959. Reset and set both phase 90° and ratio 1 to predict initial sum one. Its two waves share one simulation clock and use a chosen phase offset. It measures no clock drift, launch interval, device event, or causal correlation; the comparison only makes a changed alignment visually concrete.