advanced26 min studyLesson 160 in Engineering

Concurrency testing and schedule exploration

Build small concurrent test harnesses, enumerate meaningful executions, and state exactly what passing exploration establishes.

On this page

What you’ll understand

  • Enumerate program-order-preserving schedules for a small sequentially consistent test.
  • Distinguish schedule coverage from relaxed-memory execution coverage.
  • Design replayable concurrent tests with explicit invariants, inputs, and bounds.

A rare concurrency failure can disappear when logging is added because logging changes timing and synchronization. Running the same stress test longer may eventually expose it, but that strategy does not reveal which important executions were missed. Schedule exploration instead treats ordering decisions as controlled test inputs. A useful test is deliberately small enough to understand, has an oracle that identifies incorrect behavior, and records an execution that can be replayed. The goal is evidence with a precise scope, rather than a vague claim that many threads ran without crashing.

Start with a model you can enumerate

Take two atomic variables x and y, both initially zero. Thread A stores one to x and then loads y into a local result a. Thread B stores one to y and then loads x into b. First assume all four operations are sequentially consistent and the harness preserves each thread’s program order. A schedule is an interleaving of the two ordered pairs. Choose which two of the four positions belong to A; that gives six possible interleavings before any reduction.

The result a=b=0 is impossible under this SC model. For A to read zero, its load must precede B’s store in the single order. For B to read zero, its load must precede A’s store. Combined with the store-before-load order in each thread, those constraints form a cycle. If the implementation instead uses relaxed atomics, the zero-zero result can be allowed; enumerating only SC interleavings misses that behavior. Using unsynchronized plain C++ integers introduces data races and undefined behavior, not a valid relaxed-memory experiment.

NSC=(m+nm)=(m+n)!m! n!,(42)=6N_{\mathrm{SC}}=\binom{m+n}{m}=\frac{(m+n)!}{m!\,n!},\qquad \binom{4}{2}=6
This counts program-order-preserving interleavings of two fixed threads with m and n individually modeled atomic steps. Both counts are nonnegative integers; blocking, branching, and relaxed-memory choices require a richer model.↔ Scroll wide equations horizontally.

Make the oracle stronger than absence of a crash

A queue test can assign each produced item a unique identifier and check that accepted items are returned exactly once, respecting the required FIFO relation. Checking only a final item count misses duplication paired with loss. For a claimed linearizable object, record invocation and response intervals and test whether some legal sequential history respects real-time precedence. Overlapping calls may be ordered either way when the specification permits it; imposing one arbitrary ordering can falsely reject correct implementations.

State the harness boundaries explicitly: thread count, operation count, initial states, input values, allocation behavior, scheduling points, fairness assumptions, and language memory model. A bounded search that completes proves only that its explored model has no counterexample to that oracle within those bounds. It does not establish an unbounded progress property or cover omitted allocator reuse. Partial-order reduction can avoid schedules equivalent for the checked property, but requires a justified independence relation; it is not permission to skip inconvenient interleavings.

Preserve a replay and turn it into a regression

When exploration finds a failure, preserve both the input and the controlled execution decisions. Minimize the trace while keeping the failure: fewer threads, fewer items, and fewer preemptions make the violated invariant easier to see. A deterministic replay is especially useful for the last-item race, a missed wakeup, or a reclamation window. The harness must still model the synchronization operations actually used by the implementation; replacing real atomics with unrelated mocks can accidentally remove the bug.

Use complementary checks rather than treating one tool as universal. Dynamic race detection checks observed executions for certain races, while history checking addresses an abstract object specification. Stress tests exercise real allocation, hardware, and integration paths but provide uncertain schedule coverage. The probability lab illustrates only an assumed fixed independent chance of observing a failure per trial: predict the zero-failure bar as trial count increases. Real executions are often correlated and their failure probability is unknown, so the lab cannot certify the safety of concurrent software.

Pr⁡(no observed failure in N trials)=(1−p)N\Pr(\text{no observed failure in }N\text{ trials})=(1-p)^N
This toy expression assumes N independent trials with an identical known failure-observation probability p in [0,1]. It does not infer p from a passing stress run or establish concurrency correctness.↔ Scroll wide equations horizontally.

Enumerate the four atomic steps

For the SC store/load test above, write A for the next operation of thread A and B for the next operation of thread B. Count the six schedules and their load results, with a recording A’s load of y and b recording B’s load of x.

  1. The six schedules are AABB, ABAB, ABBA, BAAB, BABA, and BBAA. Each preserves the store-before-load order inside both threads. No thread-local reordering is silently introduced.

  2. AABB lets A load y before B stores, so (a,b)=(0,1). Symmetrically BBAA gives (1,0). The other four schedules perform both stores before either relevant load observes the other variable, so each gives (1,1).

  3. The counts are one schedule for (0,1), one for (1,0), and four for (1,1). None gives (0,0). These counts enumerate schedules, not their probabilities under a real operating-system scheduler.

The result

Six SC schedules produce three distinct result pairs. A checker permitting relaxed-memory outcomes needs additional execution choices beyond these six interleavings.

Explore the model

Interactive lab

Interactive lab · probability

A passing stress run has assumptions

Treat p as a hypothetical independent per-trial chance of observing a failure and n as the trial count. Predict the probability of zero observed failures before changing the controls.

A passing stress run has assumptionsThese are exact binomial probabilities for independent trials with one fixed success probability. Playback tours the distribution; it does not resample outcomes. Current readings: Mean · np: 5; Variance · np(1−p): 2.5; Total probability: 1. All parameters are adjustable with the labeled controls below.03581000.070.150.220.3Successes kP(X = k)k = 0 · P = 9.77e-4
Exact probabilityHighlighted outcomeMean np
Mean · np
5
Variance · np(1−p)
2.5
Total probability
1
t = 0 s

These are exact binomial probabilities for independent trials with one fixed success probability. Playback tours the distribution; it does not resample outcomes. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forThe exact binomial bars describe the stated toy assumptions. They do not measure the real failure probability, enumerate schedules, or prove a concurrent algorithm safe.

Check your understanding

A bounded checker explored all SC interleavings for two threads performing two operations each, with one initial state. Which conclusion is justified?

Try it yourself

A fixed harness has two threads with three modeled atomic steps each, no branches, and SC interleavings preserving thread-local order. How many schedules exist before reduction? If a hypothetical independent trial finds a bug with probability 0.1, what is the probability that ten such trials observe none?

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

Choose the three positions for one thread among six. For the separate toy probability, multiply ten failure-to-observe probabilities; do not equate these trials with uniform schedule sampling.

Compare with the solution

There are 6!/(3!3!)=20 interleavings. Under the explicitly independent, fixed-probability toy assumption, no observation occurs with probability 0.9^10≈0.3487, or 34.87%. The probability is not established by the schedule count, and neither calculation covers arbitrary relaxed-memory executions or larger harnesses.

Take it with you

  • Specify the oracle, bounds, and memory model before claiming exhaustive coverage.
  • SC schedule enumeration and relaxed-memory exploration cover different execution spaces.
  • Preserve minimal replayable failures and combine complementary validation methods.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.