Page pinning and direct memory access
Connect device-visible addresses, buffer lifetime, cache synchronization, and page pins without confusing CPU pointers with DMA addresses.
On this page
The question for today
What you’ll understand
- Distinguish CPU virtual, physical, and device DMA addresses.
- Explain why a pinned buffer still needs a completion and ownership protocol.
- Compare coherent and streaming DMA contracts while budgeting pinned capacity.
A device can transfer data without the CPU copying every byte, but the transfer still needs valid storage and a precise ownership agreement. Direct memory access introduces an additional address domain and an asynchronous lifetime. Pinning can keep eligible backing available for that lifetime; it does not automatically make a pointer device-readable, order descriptor publication, or prove that the device has finished using the buffer.
A CPU pointer is not a DMA address
The CPU uses virtual addresses translated through its memory system. A device may use a bus or I/O virtual address returned by the DMA mapping API, possibly translated by an IOMMU. That address need not equal either the CPU pointer or a physical address. A driver uses the platform's DMA interfaces to establish mappings, obey device addressability and alignment constraints, and check mapping failures. Casting a userspace or kernel pointer to an integer and placing it in a descriptor is not a portable DMA mapping operation.
For appropriate user-backed transfers, the kernel can obtain DMA-aware page pins through the documented pin_user_pages family rather than treating any ordinary page reference as equivalent. Long-lived registrations have stronger restrictions and can obstruct migration, compaction, or filesystem operations. Pinning a virtually contiguous user range does not establish that its frames form a single physically contiguous segment. Scatter-gather mappings or an IOMMU may provide a suitable device view, but their limits and returned segment layout must be honored. Account for what the mapping actually guarantees, not the array's visual shape in source code.
Coherency and ordering solve different problems
Coherent DMA allocation provides a documented visibility relationship between CPU and device accesses without the usual explicit streaming cache-maintenance operations. It does not make all CPU stores arrive in a desired publication order. A driver may still need the proper memory barriers before marking a descriptor ready or notifying a device. Likewise, a completion protocol must establish when device-produced contents may be consumed. Use the relevant DMA and device-ordering rules, including MMIO accessors where needed; a generic volatile annotation is not a replacement for that contract.
Streaming DMA mappings have direction and ownership rules. On platforms that require cache maintenance, mapping, unmapping, or the appropriate synchronization calls make data ready for the new owner. Reused mappings may need synchronization for CPU access and then for device access. A DMA mapping is not a lock authorizing the CPU to race with device writes. The driver must wait for the specified completion, perform the required synchronization, and only then let the next owner read or reuse the buffer. Premature unmapping, unpinning, or recycling can corrupt an unrelated future request.
Zero-copy shifts costs and obligations
Avoiding one copy can reduce bandwidth consumption, but registering buffers, managing mappings, synchronizing caches, and retaining memory can cost more than copying small transfers. Bounded reusable pools can amortize setup while making maximum retained bytes explicit. Every success, cancellation, timeout, and device-error path must eventually release the resources after device access is safely stopped. A timeout means software stopped waiting; it does not itself prove a device stopped touching memory. Shutdown and reset paths need their own completion and quiescence guarantees.
In the queue lab, press Reset, explicitly set both arrival two and service four, predict drainage of the initial four transfers in two model seconds, then Play or Step. For a separate run, Reset, set both arrival five and service four, predict one-job-per-second growth from four, then Play or Step. Rate edits alone retain elapsed time and recompute the initial-value model. Interpret each outstanding job as retaining a fixed-size registered buffer and explain why growth can retain more pinned memory. This fluid backlog is not a DMA engine or IOMMU simulator and supplies no ownership barriers or cancellation. The separate M/M/1 mean requires its stochastic assumptions.
Work it through
Budget a registered transfer pool
A bounded pool contains 128 distinct page-aligned 64 KiB buffers, backed by 4 KiB base pages. All remain registered and pinned for device use even when idle.
Each buffer spans 65,536 / 4,096 = 16 base pages. The complete pool therefore spans 128 × 16 = 2,048 distinct pages.
Pinned payload is 128 × 64 KiB = 8 MiB. Idle buffers remain part of that physical-retention budget under the stipulated registration policy.
If average transfer arrivals are 1,000 per second and a stable mean active lifetime is 0.01 seconds, Little's law predicts ten active buffers on average. That average does not justify shrinking the pool to ten without studying bursts and backpressure.
A reusable pool can reduce registration churn while retaining memory beyond its current active transfer count. Ownership and completion are still mandatory for each reuse.
The living notebook
Explore the model
Interactive lab · queue
Outstanding transfers retain buffers
Reset, explicitly set both λ=2 and μ=4, predict drainage of the initial four jobs at t=2, then Play or Step. Separately Reset, set both λ=5 and μ=4, predict growth from four, then Play or Step. Rate edits alone retain elapsed time. Equal fixed-buffer jobs illustrate retention; this model performs no DMA, cache maintenance, or completion fencing.
- Fluid backlog
- 4
- Net rate · λ − μ
- −1 /s
- M/M/1 mean residence
- 1 s
Fluid backlog starts at 4 and follows max(0, 4 + (λ − μ)t). The separate M/M/1 residence-time formula assumes Poisson arrivals, exponential service and λ < μ. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.
Look forLess copying does not remove buffer lifetimes; bounded outstanding work makes retained memory easier to budget.
Pause & reason
Check your understanding
Your turn
Try it yourself
A 9,000-byte user buffer starts 1,000 bytes into a 4 KiB page. How many base pages does it span? Does that number establish one contiguous DMA segment?
Take a moment to make a prediction or work it out on paper. Open a hint when you need one.
Give me a hint
Count the initial prefix together with the payload before taking the ceiling.
Compare with the solution
The span is 1,000 + 9,000 = 10,000 bytes from the first page boundary. Ceiling(10,000 / 4,096) is three base pages. The count says nothing about physical contiguity, IOMMU layout, device masks, or returned scatter-gather segments; the DMA mapping contract determines the usable device addresses.
Keep the idea
Take it with you
- CPU pointers, physical addresses, and device DMA addresses are distinct domains.
- Pinning, visibility, ordering, ownership, and completion solve different problems.
- Budget registered pools and all cancellation or failure paths, not just active payload.
Read further
Original sources and deeper explanations to follow at your own pace.
Optional progress, saved in this browser when available. Revisit any lesson at any time.
Lesson not marked complete.