Atomics: indivisibility is only one guarantee
Separate atomic read-modify-write behavior from publication ordering and choose memory orders from a documented proof.
On this page
The question for today
What you’ll understand
- Distinguish atomic load/store from atomic read-modify-write.
- Choose relaxed counting or release/acquire publication for explicitly different contracts.
- Explain the scope of sequential consistency and the need to verify lock-free implementation properties.
Replacing an integer with an atomic integer prevents some races but does not transform every operation around it into one indivisible transaction. Two separate atomic accesses still admit activity between them. A useful review therefore asks three questions: which individual operation is atomic, which other operations it orders, and which invariant spans multiple objects? This lesson uses counting and immutable publication to show why memory order follows the contract rather than a universal faster-versus-safer ranking.
A read-modify-write closes one particular gap
An atomic load followed by an atomic store is not the same operation as fetch-add. With a load and store, two threads can read the same old count and both write the same incremented value. The accesses themselves remain atomic, so this example can be data-race-free while its counting behavior is incorrect. An atomic read-modify-write obtains the immediately preceding value in that atomic object's modification order and writes its update as one operation, preventing that lost increment for the counter.
Relaxed fetch-add can be sufficient for a statistic when no other memory is published or consumed based on the count. It preserves atomicity and the relevant per-object ordering rules without providing a release/acquire synchronization edge for unrelated payloads. If a reader needs an exact final count, specify how it waits for all contributors to finish, for example by joining them before loading. Mid-execution observations do not imply all workers have finished their surrounding work.
Order describes relationships between evaluations
Release/acquire is useful for a directed handoff. A producer initializes a payload before its release store; a consumer that acquire-loads the corresponding published value can use the resulting synchronizes-with edge. An acquire operation does not magically collect every write made by every thread. The value it observes, its associated release or release sequence, and the surrounding sequencing determine the actual relation. A protocol involving repeated values or reused storage needs more reasoning than the simple one-shot flag.
Sequentially consistent atomics additionally participate in a specified total order for those operations. This is often a good starting point for an algorithm, but it does not make a pair of loads a snapshot or serialize whole methods. Mixing weaker operations changes what can be inferred. Invariants such as balance-plus-reservation consistency usually need a lock, a suitable combined atomic state, or a separately proved protocol. Never repair a multi-object invariant merely by attaching the strongest label to each independent field.
Optimize only after preserving the proof
A memory-order change must identify which required edges remain and which previously allowed outcomes now matter. Check the exact C++ version's operation constraints, and avoid copying assembly intuition into a portable language argument. The atomic type's lock-free property depends on its type and implementation; using std::atomic does not universally establish a lock-free algorithm. Even a lock-free primitive can sit inside a method that allocates, blocks, retries indefinitely for one thread, or invokes arbitrary user code.
The probability lab helps separate a statistical claim from a correctness guarantee. For fixed trial count, increase success probability and predict the changed mean and variance of the exact binomial distribution. Then ask why an unobserved failure in many test runs is still not a memory-model proof: executions need not be independent trials with a fixed failure probability. The lab does not execute atomic instructions or estimate their correctness. Use it to question the sampling assumption before quoting a reliability percentage from stress tests.
Work it through
Two atomic increments that are not fetch-add
An unsigned atomic counter starts at seven. Two threads each perform a separate load, add one locally, then store. Compare a legal lost-update history with two fetch-add operations, assuming no overflow or other writes.
In the separate-operation history, thread A loads seven and thread B loads seven before either store. Both compute eight locally. A stores eight and B then stores eight, leaving eight although two increment requests ran.
With fetch-add, one read-modify-write reads seven and writes eight. The other reads the immediately preceding eight in modification order and writes nine. Which thread gets which returned value is not predetermined.
Join both workers before observing the final statistic. Relaxed ordering can suffice for this isolated count, but it does not publish payload arrays that the workers might also have written. State that additional contract separately.
The separate load/store design may end at eight; fetch-add gives nine under the stated conditions. Stronger ordering on separate accesses does not itself combine them into an atomic increment.
The living notebook
Explore the model
Interactive lab · probability
Sampling is not an atomicity proof
Vary success probability p and integer trial count n in the exact binomial model. Predict mean np before inspecting it. The diagram assumes independent equal-probability trials and does not model atomics, weak memory, or a stress test's schedules.
- Mean · np
- 5
- Variance · np(1−p)
- 2.5
- Total probability
- 1
These are exact binomial probabilities for independent trials with one fixed success probability. Playback tours the distribution; it does not resample outcomes. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.
Look forA statistical model answers only the question its trial assumptions define; atomic correctness requires a language-level argument.
Pause & reason
Check your understanding
Your turn
Try it yourself
Four workers each execute 25,000 relaxed fetch-add increments on an initially zero unsigned 64-bit statistic, then all are joined. What final count is expected, and what extra conclusion about four ordinary output buffers is unjustified from the count alone?
Take a moment to make a prediction or work it out on paper. Open a hint when you need one.
Give me a hint
Separate exact arithmetic for one counter from the protocol used to access other objects.
Compare with the solution
The count is 100,000 because each successful atomic read-modify-write contributes one, there are no other writes, and this is far below wraparound. The relaxed operations alone do not establish publication edges for the output buffers. In this specific scenario, successful thread joins can supply the needed completion synchronization for worker writes, provided lifetimes and conflicting-access rules are satisfied. If a different observer reads the buffers merely after seeing the relaxed count reach 100,000, that observer needs a separately justified handoff.
Keep the idea
Take it with you
- Atomic load plus atomic store is not an atomic read-modify-write.
- Relaxed atomicity and release/acquire publication solve different problems.
- Sequential consistency and a lock-free primitive do not prove a whole-method invariant or progress bound.
Read further
Original sources and deeper explanations to follow at your own pace.
Optional progress, saved in this browser when available. Revisit any lesson at any time.
Lesson not marked complete.