advanced24 min studyLesson 172 in Engineering

Allocation decisions and the cost of fragmented space

Distinguish free-space fragmentation from file fragmentation, explain delayed allocation, and evaluate placement against actual workload costs.

On this page

What you’ll understand

  • Separate internal allocation slack, fragmented free space, and fragmented files.
  • Explain how delayed allocation and preallocation alter placement opportunities.
  • Estimate a workload-specific extent overhead without assuming an SSD behaves like a disk.

A filesystem can have plenty of free bytes and still struggle to find a suitably sized run, reserve metadata, or satisfy a particular operation. Allocation is a decision about shape, location, timing, and future changes as well as total capacity. Fragmentation names several different problems that require different evidence. We will build a small accounting model, then ask when its costs correspond to the device and access pattern we actually operate.

Three kinds of wasted opportunity

Internal fragmentation is unused space within allocated units. If a simplified filesystem dedicates whole blocks to each small file, the final block may contain slack. Free-space fragmentation instead describes how unused blocks are split into runs. File fragmentation describes one file's contents spread across multiple mappings. A volume with many scattered free blocks can still store a new file, yet require more extents. Do not add these quantities as though they were the same percentage; each uses a different denominator and describes a different mechanism.

Application access order matters. A large sequential scan can benefit from long contiguous mappings because they reduce mapping work and support larger requests. An application that reads isolated random blocks may receive less benefit from making every file sequential. On a rotating disk, movement between distant requests adds a mechanical cost. An SSD has no seek arm, but request count, controller parallelism, translation, and transfer size still matter. Therefore a fragmented-file count is a clue, not a direct measurement of elapsed-time loss across all devices.

A=∑i=1mB⌈siB⌉,W=A−∑i=1msiA=\sum_{i=1}^{m}B\left\lceil\frac{s_i}{B}\right\rceil,\qquad W=A-\sum_{i=1}^{m}s_i
For m nonempty files of size s_i bytes with dedicated whole blocks of B bytes, A is allocated payload and W is internal slack. This simplified model excludes metadata, sparse holes, inline data, compression, and shared blocks.↔ Scroll wide equations horizontally.

Delay a decision or reserve its space

Ext4 delayed allocation can postpone choosing data-block locations until dirty data is written back. Seeing a larger span of writes together can improve placement compared with allocating independently for every tiny arrival. The tradeoff is that accepting buffered writes is not equivalent to having permanently stored them, and allocation or I/O errors still require handling. A reservation, an allocated mapping, and durable contents are different milestones. Record which milestone an API promises before using a free-space observation as evidence that a checkpoint can be published.

Preallocation can establish space before a stream of later writes and reduce some fragmentation risks, but its precise semantics depend on the operation and filesystem. It can also reserve capacity that another task needed. Workloads that grow many files concurrently, retain snapshots, or rewrite small regions can defeat a placement strategy that looked effective on an empty volume. Keep room for metadata and future updates rather than targeting a universal safe free-space percentage. Capacity plans should include measured behavior near their intended operating range, not merely a successful initial allocation.

Tscan≈Dv+EτT_{\rm scan}\approx\frac{D}{v}+E\tau
This illustrative sequential-scan model moves D bytes at effective transfer rate v bytes per second and pays an assumed overhead tau seconds for each of E extent transitions. It is useful only when those costs are approximately separable; it is not a universal SSD or disk law.↔ Scroll wide equations horizontally.

Measure the mechanism before reorganizing data

Compare a representative operation before and after a placement change while controlling file contents, cache state, concurrency, and available capacity. Observe both extent layout and block-request behavior. A faster second run might simply be cached. A defragmentation operation can itself generate heavy I/O, alter shared-extent storage consumption on some copy-on-write filesystems, or compete with user traffic. Treat it as a workload with a purpose and a rollback or recovery plan, rather than a cosmetic way to reduce a counter.

The quadratic lab is a deliberate scaling analogy for metadata-heavy all-pairs inspection. Interpret positive x as normalized extent count and positive a as work per pair for a naive comparison procedure. Double x and predict fourfold y before moving the control. This illustrates why a poor diagnostic algorithm becomes expensive as mappings multiply; it does not predict fragmentation or defragment a filesystem. A practical investigation should distinguish the cost of the layout from the cost of the tool inspecting it, and favor indexed or streaming analysis when appropriate.

Separate byte waste from scan overhead

One thousand files each hold 5,000 bytes on a simplified 4,096-byte-block filesystem. Separately, a 200 MB sequential scan transfers at 100 MB/s and crosses 80 extent boundaries at an assumed 0.5 ms each. Use decimal MB.

  1. Each small file needs two blocks: 8,192 bytes allocated for 5,000 bytes of contents. One thousand files allocate 8,192,000 bytes, with 3,192,000 bytes of internal slack.

  2. The scan's transfer term is 200/100 = 2 seconds. Its assumed transition term is 80 × 0.0005 = 0.04 seconds, giving 2.04 seconds in the model.

  3. If transitions fall to ten while transfer rate stays fixed, the modeled time is 2.005 seconds. That improvement says nothing about the small-file slack, which requires a different representation or allocation policy.

The result

The two examples quantify different costs: 3,192,000 bytes of slack and 35 milliseconds of modeled transition savings. Neither justifies a blanket claim that every fragmented workload is slow.

Explore the model

Interactive lab

Interactive lab · function

When an inspection algorithm scales poorly

Set positive coefficient a and position x in y=ax², treating x as normalized extent count in an illustrative all-pairs diagnostic. Predict the output change when x doubles. The lab models quadratic work, not free-space placement, real scan latency, or defragmentation.

When an inspection algorithm scales poorlyThis lab uses only y = ax². Playback traces the curve within the fixed integration interval from 0 to x; it does not estimate numerical error or change the chosen point. Current readings: y at chosen x: 0.608; Tangent slope · 2ax: 1.35; Signed area · ax³/3: 0.182. All parameters are adjustable with the labeled controls below.−2.4−1.201.22.4−9−4.504.59xy(x, ax²)
y = ax²TangentSigned integral
y at chosen x
0.608
Tangent slope · 2ax
1.35
Signed area · ax³/3
0.182
t = 0 s

This lab uses only y = ax². Playback traces the curve within the fixed integration interval from 0 to x; it does not estimate numerical error or change the chosen point. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forMeasure layout cost separately from diagnostic cost; neither a fragmentation percentage nor a quadratic analogy identifies a device bottleneck.

Check your understanding

A volume has 40 GiB free but no free run larger than 8 MiB. Which conclusion is justified for an ordinary extent-based filesystem that can allocate multiple extents?

Try it yourself

A 600 MB scan achieves 150 MB/s. Its 200 extent transitions each add an assumed 1 ms. Estimate modeled time, then repeat for twenty transitions without changing transfer rate.

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

Add a transfer term in seconds and a transition term in seconds for each layout.

Compare with the solution

Transfer takes 600/150 = 4 seconds. Two hundred transitions add 0.2 seconds, giving 4.2 seconds; twenty add 0.02 seconds, giving 4.02 seconds. The modeled saving is 0.18 seconds. Real results need measurement because transition costs may overlap, cache behavior can change, and SSD placement is not a simple mechanical-seek problem.

Take it with you

  • Internal slack, free-space fragmentation, and file fragmentation are separate measurements.
  • Delayed allocation and preallocation change placement opportunities, not the meaning of durability.
  • Evaluate reorganization against representative requests, cache state, and its own I/O cost.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.