greatitself.
Continue
research28 min studyLesson 993 in Engineering

Separate coherent access from cheap access

Read CXL device capabilities and region semantics before modeling latency, sharing, synchronization, or offload.

Before you begin
On this page

The question for today

What you’ll understand

  • Distinguish CXL.io, CXL.cache, and CXL.mem roles.
  • Compute a memory-level-parallelism bandwidth ceiling.
  • Explain why coherence does not replace software synchronization or workload measurement.

A coherent memory link can let processors and devices share addressable data with fewer explicit copies, but it does not make remote memory local or make every accelerator capable of every CXL protocol. A memory expander and an accelerator with coherent caching have different roles. Start with the actual host, device, firmware, topology, and operating-system capabilities, then identify which address regions are shared under which rules. Only after that semantic inventory should latency and offload costs enter the model. The numeric link examples here are fictional and describe no commercial CXL product.

Name the protocol and device role

The CXL 3.2 revision 1.0 architecture distinguishes configuration and I/O through CXL.io, device caching of host memory through CXL.cache, and host access to device-attached memory through CXL.mem. Its Type 1, Type 2, and Type 3 profiles combine these protocols differently. In particular, a Type 3 memory device does not imply a CXL.cache request path. Device-attached memory can also have region-specific coherence management, so the host and device must agree on that region’s supported mode.

A specification revision is a reference for semantics, not a discovery API proving that a platform implements every feature discussed in that revision. Check negotiated link properties, device capabilities, address decoders, memory-region configuration, and software enablement. Switches, pooling, peer access, and sharing need their own support and management conditions. A deployment proposal should list required capabilities and observed platform evidence rather than infer them from a product page that says CXL-compatible.

Tdependent≈nL,Buseful≤min⁡(Bpayload,msL)T_{\rm dependent}\approx nL,\qquad B_{\rm useful}\leq\min\left(B_{\rm payload},\frac{m s}{L}\right)
The latency estimate assumes n serialized dependent misses of latency L seconds. The separate bandwidth bound assumes at most m outstanding independent transactions, each delivering s useful bytes, with mean occupancy time L. Payload bandwidth and all rates use bytes/s; real contention and protocol traffic can tighten the bound.↔ Scroll wide equations horizontally.

Coherence has a scope and an ownership cost

Coherence coordinates copies of an individual memory location within its supported domain. It does not automatically establish the application’s required ordering between a payload and a ready flag, make a compound structure atomic, or persist data across power loss. Use the platform’s documented atomic and synchronization operations with the required scope. A pointer being dereferenceable on both participants does not prove that a racy producer-consumer algorithm is correct.

Shared writable cache lines can also move ownership repeatedly between participants. Separate frequently written metadata from unrelated fields and consider phased ownership when the workload permits it. Fine-grained shared access may save an explicit bulk copy but introduce many small coherence transactions. Bulk-local computation may win when reuse is high. Compare both approaches with the same consistency and completion requirements, including any transitions, flushes, and synchronization demanded by the supported memory mode.

Toffload=Tsubmit+Taccess+Tcompute+TpublishT_{\rm offload}=T_{\rm submit}+T_{\rm access}+T_{\rm compute}+T_{\rm publish}
This deliberately serial decomposition uses seconds and assigns each cost once. Taccess includes the actual coherent or explicit-transfer path; Tpublish includes required completion/ordering. Overlap needs a separate dependency schedule rather than simply removing a term.↔ Scroll wide equations horizontally.

Measure access patterns rather than a version label

Dependent pointer chasing probes latency while many independent streams can expose bandwidth and queueing. A sequential scan, random embedding lookup, and repeated shared-counter update stress different mechanisms. Measure placement and caching explicitly, because a benchmark that repeatedly hits a CPU cache does not characterize device-memory miss latency. Report useful bytes separately from link traffic when coherence metadata or full-line transfers amplify the physical movement.

For offload, retain the CPU baseline and include data residency, synchronization, errors, and restart behavior. Coherent access may simplify a programming interface without improving every workload’s runtime. Capacity expansion can still be valuable even when access is slower, provided the alternative is spilling to an even slower tier. State that tradeoff directly. Neither coherence nor a higher link transfer rate supplies a universal latency, energy, or application-speedup guarantee.

Work it through

Concurrency limits a fictional memory link

Assume a fictional memory path has payload ceiling 32 decimal GB/s, mean transaction occupancy 300 ns, and 64 useful bytes per transaction. At most 64 independent transactions can be outstanding. Compute the concurrency ceiling, then consider 256 outstanding transactions and 1000 serialized dependent misses.

  1. With m=64, the occupancy bound is 64×64/(300×10⁻⁹)=13.6533×10⁹ bytes/s, or about 13.65 GB/s. It is below the 32 GB/s payload ceiling.

  2. With m=256, the occupancy bound rises to about 54.61 GB/s, but useful rate remains bounded by min(32,54.61)=32 GB/s before other overheads.

  3. A serialized chain of 1000 dependent misses takes approximately 1000×300 ns=300 microseconds under the constant-latency assumption. Extra potential concurrency does not help that chain if each address depends on the preceding result.

The result

Latency, available concurrency, and payload bandwidth constrain different access patterns. These synthetic numbers establish no measured result for CXL or any specific device.

The living notebook

Explore the model

Interactive lab

Interactive lab · cache

Check locality before blaming a remote link

Reset, set Cache capacity C to 4 and Request stride s to 4. Predict 4 initial misses and 12 hits among 16 requests; one Step adds a hit. Reset, explicitly set C=2 and s=4, then Step: predict no hits among 17 requests. The fully associative LRU trace caps at 4096 requests. It models reuse only, with no CXL coherence states, snoops, memory latency, or shared writes.

Check locality before blaming a remote linkA fully associative LRU cache starts empty and requests address (i·s) mod 16. Hits move an address to the newest position; misses evict the oldest when full. Playback starts with 16 requests and adds two per second, capped at 4096 explicitly simulated requests. This toy stream is not a processor-cache benchmark. Current readings: Hits / requests: 0 / 16; Hit rate: 0%; Address-stream period: 16 addresses. All parameters are adjustable with the labeled controls below.Fully associative LRU cache after 16 deterministic requests: 0 hits and 16 misses.Most recent 16 requestsaddress = i × 3 mod 160M3M6M9M12M15M2M5M8M11M14M1M4M7M10M13MCurrent cache order · oldest → newest471013Latest address 13 · miss: inserted as newestToy address stream · period 16 · starts empty
Cached addressMiss / evictionMost recent request
Hits / requests
0 / 16
Hit rate
0%
Address-stream period
16 addresses
t = 0 s

A fully associative LRU cache starts empty and requests address (i·s) mod 16. Hits move an address to the newest position; misses evict the oldest when full. Playback starts with 16 requests and adds two per second, capped at 4096 explicitly simulated requests. This toy stream is not a processor-cache benchmark. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forCache locality and coherence semantics are different parts of the access path.

Pause & reason

Check your understanding

Which claim is valid for a CXL Type 3 memory device in the referenced architecture?

Your turn

Try it yourself

A fictional path has 64-byte useful transfers, mean occupancy 400 ns, and a target useful rate of 16 decimal GB/s. What minimum outstanding-transaction count is required by the occupancy bound? Is that sufficient to guarantee the target rate?

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

Rearrange B≤ms/L and round the required count upward.

Compare with the solution

The requirement is m≥BL/s=(16×10⁹)(400×10⁻⁹)/64=100 outstanding transactions. It is necessary under this simple occupancy model but not sufficient: payload capacity, queueing, host limits, access dependencies, and protocol overhead may prevent the rate.

Keep the idea

Take it with you

  • Inspect protocols, region modes, and implemented capabilities independently.
  • Coherence does not replace synchronization, persistence, or performance evidence.
  • Model latency, concurrency, and useful-byte bandwidth for the actual access pattern.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.