advanced27 min studyLesson 516 in Engineering

Order observable effects even when tensor values are independent

Separate value dependencies, mutable state, pseudorandom keys, I/O, and speculation safety when deciding which operations may move or repeat.

On this page

What you’ll understand

  • Identify effect dependencies absent from a tensor-only use-def graph.
  • Explain how explicit state or tokens can preserve required order.
  • Distinguish pure keyed randomness from implicit mutable generator state.

Two tensor operations can have no shared numerical input and still be unsafe to reorder. One may update a counter, consume generator state, print a diagnostic, or communicate with an external service. Those effects belong to program behavior even when the returned tensor is unchanged. Compiler analysis must identify the required ordering and duplication rules at the relevant execution boundary. An ordinary Python action during tracing is also different from an action represented in the executable: observing it once while compiling does not mean it will run once per request.

Represent the state transition that a read-only graph omits

Consider an operation that returns the current integer counter and then increments it modulo eight. Two calls starting from zero must return zero and one, leaving the state at two. If a compiler incorrectly treats this operation as a pure constant-producing node, common-subexpression elimination could replace the second call with the first result. The numerical tensor graph may then look simpler while the observable sequence has changed. A state input and output make the dependency explicit rather than asking the optimizer to infer hidden global behavior.

I/O introduces similar obligations, with additional external uncertainty. Deleting a log operation because its tensor result is unused can remove a promised audit event. Duplicating a request can repeat a side effect. Moving an operation above a conditional can execute it when the source program did not. MLIR's side-effect and speculation rationale distinguishes effect information from permission to speculate: even an operation without ordinary memory writes may trap, have undefined behavior for some inputs, or fail to terminate under its semantics.

(ri,si+1)=(si,(si+1) mod 8),s0=0(r_i,s_{i+1})=(s_i,(s_i+1)\bmod8),\qquad s_0=0
Here s is explicit dimensionless counter state and r the returned value before the update. The transition is deterministic and sequential. It is an illustrative state contract, not an atomic concurrent counter, cryptographic random generator, or description of every framework's state representation.↔ Scroll wide equations horizontally.

Randomness needs an explicit contract too

With an implicit mutable pseudorandom generator, two draws can depend on the order in which generator state is consumed. Reordering or recomputing them may change both outputs and later state. With a pure keyed design, the key is an explicit input: using the same key and same distribution operation can intentionally reproduce the same result. Distinct logical draws require the documented key-derivation or splitting discipline. Reusing a key is not the same as advancing an implicit global generator once more.

JAX documentation describes explicit keys and effect-ordering mechanisms, but this is not a claim that every framework operation is pure or that arbitrary Python effects are supported inside every transform. Separate host tracing effects, supported runtime effects, and external state. For checkpointing or recomputation, ask which operations may run again and what state must be restored. Matching the final loss once is a weak oracle when the transformation has silently consumed a different amount of state or repeated an external action.

k1,k2=split⁡(k),u1=f(k1),u2=f(k2)k_1,k_2=\operatorname{split}(k),\qquad u_1=f(k_1),\quad u_2=f(k_2)
This schematic pure keyed interface assumes the framework's documented split and distribution semantics. The symbols are dimensionless keys and samples with a declared distribution, not an assertion of cryptographic security or a proof of statistical independence for arbitrary user-defined f.↔ Scroll wide equations horizontally.

Order only what the contract requires, then test the trace

An ordering token can carry an otherwise absent dependency between effectful operations. It does not magically make an external service transactional or impose a universal order across independent processes. The scope of runtime synchronization and compiler ordering must be stated. An analysis can permit disjoint effects to proceed independently when their contracts allow it, while preserving required order for writes to one resource. This is more precise than either treating all tensor nodes as pure or serializing every operation without examining its effects.

The modular lab makes one explicit state sequence inspectable. Reset, set both modulus m=8 and additive step a=3, predict residues zero, three, six, one, four, then take four Steps. Reset and set both m=8,a=1 to compare a different transition contract. Each Step advances one addition and the initial residue is zero. The lab does not generate random numbers, emit I/O, model concurrency, or execute effect tokens; it only shows why a specified sequence of state reads should not be silently replaced by repeated reads of the initial state.

Preserve read-before-update results

A fictional operation returns counter state s, then updates s←(s+3) mod 8. Start with s=0 and call it four times sequentially. All arithmetic is exact and there are no concurrent writers.

  1. First return zero and update to three. The second call returns three and updates to six, so the first two observed results are [0,3].

  2. The third call returns six and updates to one because nine modulo eight is one. The fourth returns one and updates to four.

  3. The required output sequence is [0,3,6,1] and final state four. Replacing all calls with one common result would produce [0,0,0,0] and cannot preserve this stateful contract.

The result

Both the returned sequence and final state are necessary test observations. A tensor-only output test that ignores state can miss incorrect deletion, duplication, or reordering.

Explore the model

Interactive lab

Interactive lab · modular

Follow an explicitly specified state transition

Reset → set both m=8,a=3; predict residues 0,3,6,1,4, then take four Steps. Reset → set both m=8,a=1 for a separate comparison. The additive orbit is deterministic and has no RNG, I/O, concurrency, or compiler effect-token execution.

Follow an explicitly specified state transitionAddition modulo m starts at zero: rₙ = na mod m. Playback advances two additions per second. The orbit has period m/gcd(m,a), including period one for a = 0. This is not a cryptographic security test or a primality algorithm. Current readings: Residue r0: 0; Orbit period · m/gcd(m,a): 12; Distinct residues visited: 1 of 12. All parameters are adjustable with the labeled controls below.Repeated addition modulo 12: the current residue is 0, and the orbit period is 12.01234567891011+ 5mod 12An additive orbitn = 0rₙ = 0Period = 12Cycle starts at 0:0 → 5 → 10 → 3 →8 → 1 → 6 → 11 →4 → 9 → 2 → 7 → 0Exact residues · two additions per second
Additive orbitCurrent residue / next edge
Residue r0
0
Orbit period · m/gcd(m,a)
12
Distinct residues visited
1 of 12
t = 0 s

Addition modulo m starts at zero: rₙ = na mod m. Playback advances two additions per second. The orbit has period m/gcd(m,a), including period one for a = 0. This is not a cryptographic security test or a primality algorithm. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forState transitions create observable dependencies beyond an ordinary pure tensor graph.

Check your understanding

An ordinary Python print occurs while a pure array function is traced for JIT compilation. What can be concluded about later executable calls?

Try it yourself

A function returns a tensor and appends one record to a required event sink. A transformation returns identical tensors but appends two records because it recomputes the whole function. Is tensor equality sufficient, and what fixture should expose the failure?

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

Include the promised external observation in the oracle and control the sink so the expected event count is unambiguous.

Compare with the solution

Tensor equality is insufficient because one promised event has become two. Use a controlled sink recording event identity, count, and any required order; run a known number of calls and compare both tensor outputs and event observations. If recomputation is needed, isolate the pure region or use a supported effect-preserving arrangement. A production sink's retries and delivery guarantees require their own model, so the fixture should state its simple exact-count assumptions.

Take it with you

  • Effects can create dependencies absent from tensor dataflow.
  • Explicit state and ordering tokens preserve scoped contracts rather than universal external guarantees.
  • Trace-time Python actions, runtime effects, and keyed randomness need distinct oracles.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.