advanced23 min studyLesson 144 in Engineering

Memory-mapped file semantics

Separate mapping visibility, synchronization, file extent, and durability when ordinary loads and stores access file-backed data.

On this page

What you’ll understand

  • Compare MAP_SHARED with MAP_PRIVATE for file updates.
  • Calculate page-aligned mapping offsets and accessed byte ranges.
  • Explain why visibility, atomicity, and durable storage are separate requirements.

Mapping a file allows a program to address file-backed data through memory operations. That can simplify access and share cached data, but it does not turn memory stores into transactions or abolish storage failures. A sound design specifies who may change the file, how readers coordinate, which bytes exist, and what must survive a crash before choosing the mapping flags.

The mapping type decides where writes belong

A shared file mapping makes updates available through mappings of the same underlying region and allows those updates to reach the file. A private mapping instead gives the process copy-on-write modifications that are not written back to that file. Private does not mean a permanently frozen snapshot of arbitrary concurrent external file changes; visibility of later changes to the underlying object has limits in the documented interface. If a reader needs a stable snapshot, use an explicit versioning or ownership protocol rather than inferring one from the word private.

The mapping offset normally must be aligned to the system page size. To access an unaligned logical file position, map from the preceding page boundary and add the position's offset within that page to the returned pointer. Include that prefix when sizing the mapping. A mapping can remain valid after its original file descriptor is closed, because the mapping holds its own reference. None of this makes a stale application pointer safe after munmap or authorizes access beyond the mapped length. Pointer lifetime still needs an owner.

omap=P⌊oP⌋,d=o−omap,Lmap=d+Lo_{\rm map}=P\left\lfloor\frac{o}{P}\right\rfloor,\qquad d=o-o_{\rm map},\qquad L_{\rm map}=d+L
o is the desired file byte offset, L the desired byte length, and P the page-size alignment in bytes. d is the prefix skipped in the returned mapping. The requested interval must exist and the mapping call must succeed; the formula alone does not establish either.↔ Scroll wide equations horizontally.

Visibility is not a transaction or a crash guarantee

Two processes sharing a page need synchronization that is valid for shared-memory communication. A reader must not observe a half-published multi-field record simply because the bytes are in a shared mapping. Language memory-ordering rules, process-shared synchronization, record formats, and recovery rules matter. Ordinary CPU cache coherence is not a replacement for these rules. A convenient design can write a complete new version and publish a reference using an appropriate protocol; a different design may require a journal or checksums. The chosen mechanism must match the actual concurrency and crash model.

Dirty shared pages can remain in the page cache before writeback. An explicit synchronization operation such as msync with MS_SYNC requests completion for the specified mapped range and can fail; check its result. It does not automatically supply multi-page transactional atomicity. File creation, extension, naming, and directory persistence may need additional filesystem-specific and fsync-related steps. Do not use munmap or close as a substitute for a documented durability protocol. A device that lies about flush completion also changes the failure model, independently of how the application addressed the bytes.

Twriteback≥DBT_{\rm writeback}\ge\frac{D}{B}
D is bytes that must reach a bottleneck and B is its sustained bytes-per-second bandwidth. The lower bound excludes flush latency, metadata, write amplification, and interference. Shared-mapping visibility can occur before this durability work completes.↔ Scroll wide equations horizontally.

A mapped address can outlive its backing extent

If another actor truncates a mapped file, a later access to a page beyond its current end can raise SIGBUS on Linux. A mapping is not a reservation preventing file-size changes by somebody else. The partial page at the end also has special rules; do not use bytes past the file's logical end as persistent scratch space. For safe data access, coordinate file replacement and truncation with the mapping users, validate record lengths against a stable file version, and plan how corrupt or incomplete records are rejected.

Explore writeback demand in two separate queue-lab runs. Press Reset, explicitly set both arrival three and service five, predict the initial four-job backlog drains after two model seconds, then Play or Step. Reset again, set both arrival five and service four, predict one-job-per-second growth from four, then Play or Step. Editing rates without Reset retains elapsed time and recomputes the initial-value model rather than preserving a live backlog. Treat jobs as equal normalized dirty-data batches solely to reason about imbalance. This does not simulate dirtying thresholds, msync completion, filesystem ordering, or crashes. Ask which acknowledgment would prove durable completion beyond finishing stores.

Map an unaligned record range

A file is at least 20,000 bytes long. Read 6,000 bytes beginning at file offset 5,000 using ordinary mmap on a system with 4 KiB page alignment.

  1. Round the file offset down to 4,096. The desired first byte is 5,000 − 4,096 = 904 bytes after the returned mapping base.

  2. Request a mapping length of 904 + 6,000 = 6,904 bytes. The application interval is file bytes 5,000 through 10,999 inclusive; all those bytes exist under the stated stable-file assumption.

  3. Use the returned base plus 904 for the first record byte, retain the original base and requested length for unmapping, and keep the mapped lifetime valid throughout access. A concurrently truncated file would invalidate the stable-extent assumption.

The result

Page alignment constrains the mapping start, not the logical record start. Sharing, synchronization, and durability are separate decisions from this address calculation.

Explore the model

Interactive lab

Interactive lab · queue

Dirty-data demand and writeback capacity

Interpret jobs as normalized dirty-data batches. Reset, explicitly set both λ=3 and μ=5, predict drainage from four at t=2, then Play or Step. For a separate run, Reset, set both λ=5 and μ=4, predict growth from four, then Play or Step. Rate edits alone retain elapsed time. This fluid model supplies no filesystem ordering or durable acknowledgment.

Dirty-data demand and writeback capacityFluid backlog starts at 4 and follows max(0, 4 + (λ − μ)t). The separate M/M/1 residence-time formula assumes Poisson arrivals, exponential service and λ < μ. Current readings: Fluid backlog: 4; Net rate · λ − μ: −1 /s; M/M/1 mean residence: 1 s. All parameters are adjustable with the labeled controls below.02.557.51001.534.56Elapsed time · sFluid backlogλ < μ
Fluid backlogCurrent time
Fluid backlog
4
Net rate · λ − μ
−1 /s
M/M/1 mean residence
1 s
t = 0 s

Fluid backlog starts at 4 and follows max(0, 4 + (λ − μ)t). The separate M/M/1 residence-time formula assumes Poisson arrivals, exponential service and λ < μ. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forFinishing a store and completing the storage work needed for durability are different events.

Check your understanding

A process modifies a MAP_PRIVATE mapping of a file and then requests synchronization. Which claim follows from the mapping type?

Try it yourself

With 4 KiB alignment, map 100 bytes beginning at file offset 8,200. Give the mapping offset, prefix, and minimum requested length. State one correctness requirement beyond those numbers.

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

8,192 is the previous page boundary.

Compare with the solution

The mapping offset is 8,192 bytes, the prefix is eight bytes, and the requested length is 108 bytes. The file must contain the full requested logical interval and remain stable under the access protocol; the mapping must succeed and remain alive. If writes are shared, the program also needs appropriate synchronization and an explicit durability plan.

Take it with you

  • MAP_SHARED and MAP_PRIVATE define different update semantics.
  • Address alignment, stable file extent, and mapped-pointer lifetime are separate obligations.
  • Shared visibility does not prove transactional atomicity or crash durability.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.