advanced24 min studyLesson 280 in Engineering

Index maintenance: count the work behind one logical update

Define write-amplification boundaries, account for MVCC and HOT behavior, and compare index benefits with sustained maintenance cost.

On this page

What you’ll understand

  • Define a write-amplification ratio with matching measurement boundaries.
  • Explain why one logical update can affect several indexes and obsolete versions.
  • Evaluate cleanup, rebuilds, and added indexes against a representative read-write workload.

An index makes some reads cheaper by doing and storing extra work elsewhere. Inserts, updates, deletes, cleanup, and rebuilds all contribute to that bill. A useful write-amplification discussion states the logical operation and the layer where bytes or events are counted. Otherwise a low ratio at one boundary can coexist with expensive maintenance hidden in another.

Choose a numerator and denominator deliberately

Logical user payload, database page writes, write-ahead log bytes, filesystem writes, and device-internal writes describe different layers. Define a ratio using a stated denominator, such as logical changed payload bytes over a time interval, and a stated numerator, such as database-generated data and log bytes in that same interval. Include or exclude background cleanup explicitly. A page dirtied five times may be written once before eviction, so counting dirty events as if each were a complete physical page write overstates some workloads and misses other logging behavior.

Avoid adding overlapping counters. If a storage measurement already includes log writes, adding the separately measured log total double-counts them. Device-level amplification can include garbage collection inside flash storage that the database does not report. Conversely, a benchmark ending before dirty pages or deferred cleanup are flushed can hide later work. Compare equivalent durability, checkpoint, cache-warmth, and cleanup conditions. Ratios with a tiny logical-byte denominator can be large without implying an implementation bug; small updates naturally interact with page- and log-oriented mechanisms.

Awrite=WmeasuredUlogical,Ulogical>0A_{\rm write}=\frac{W_{\rm measured}}{U_{\rm logical}},\qquad U_{\rm logical}>0
W_measured and U_logical are byte counts for the same stated interval and workload. A_write is dimensionless and meaningful only with a precise numerator layer, durability boundary, and treatment of deferred/background work. It does not automatically include device-internal amplification.↔ Scroll wide equations horizontally.

Versioning and index dependencies determine maintenance

A logical row update can create a new physical row version. Indexes referencing physical versions may need new entries even when some indexed key values did not logically change. PostgreSQL's HOT optimization can avoid new entries in ordinary indexes under documented conditions, including suitable same-page space and restrictions on modified indexed columns. Included columns count as index dependencies for this decision. PostgreSQL 18 has a specific exception for summarizing indexes such as BRIN, so the blanket statement that any indexed-column update always forbids HOT is too broad.

Deleting a row does not always erase every old index entry immediately: active snapshots and recovery rules constrain reclamation. Vacuum, bottom-up index cleanup, deduplication where supported, and page recycling can reduce obsolete versions or avoid splits. These mechanisms affect different stages and do not guarantee the file instantly shrinks. A rebuild can produce a compact structure while temporarily consuming extra storage and I/O. Concurrent rebuild or construction has its own locking, validation, and failure behavior; selecting the concurrent option does not make the operation free of resource or availability consequences.

Wˉ=Wtable+Wlog+∑j=1JWindex,j+Wcleanup\bar{W}=W_{\rm table}+W_{\rm log}+\sum_{j=1}^{J}W_{{\rm index},j}+W_{\rm cleanup}
This additive accounting model uses disjoint byte categories per logical operation averaged over a complete measurement interval. The categories must not overlap, and cleanup must be assigned consistently. It is bookkeeping for the stated database boundary, not a universal formula for physical flash writes or every engine's update protocol.↔ Scroll wide equations horizontally.

Select indexes as a workload portfolio

For each proposed index, identify the query family it accelerates and measure how often that family occurs. Compare the saved scan, sort, and row-fetch work with extra writes, storage, cache pressure, and recovery volume. An apparently redundant index may still enforce a required uniqueness constraint or serve a distinct ordering; removing it requires understanding those semantics. Conversely, several wide indexes added for rare reports can penalize every frequent update. Test representative bursts and skew, not only steady read-only queries after a convenient bulk load.

The queue lab illustrates an explicit maintenance-rate budget. Reset, set λ=4 and μ=3, predict one additional queued work unit per second, then Play or Step. Separately Reset, set λ=2 and μ=3, predict the restored four-unit initial backlog drains in four seconds, then Play or Step. The instrument has no indexes, WAL, vacuum, or storage device. Associate each work unit with a clearly stated synthetic maintenance quantity and explain why an ever-growing queue cannot be fixed simply by deferring the same required work indefinitely.

Keep the byte categories disjoint

Over a complete fictional workload interval, logical changed user payload totals 10 MB. Disjoint database-generated byte counters report 20 MB table data, 30 MB index data, 25 MB WAL, and 5 MB cleanup not included in the other categories.

  1. Add only the disjoint measured categories: 20+30+25+5=80 MB. All values use the same decimal megabyte unit and interval.

  2. Divide by logical payload 10 MB to obtain database-boundary write amplification 80/10=8. The unit cancels, and the ratio includes the stated cleanup.

  3. The result says nothing directly about device-internal flash writes or request latency. Compare a changed index portfolio only after preserving durability and including its corresponding deferred work.

The result

The measured ratio is eight at the specified database boundary. A different layer or incomplete interval would answer a different question.

Explore the model

Interactive lab

Interactive lab · queue

Deferred maintenance still needs service

Reset, set λ=4 and μ=3, predict backlog growth of one work unit per second, then Play or Step. Separately Reset, set λ=2 and μ=3, predict the four-unit initial backlog drains in four seconds, then Play or Step. This fluid work budget does not simulate index updates, WAL, cleanup, or physical writes.

Deferred maintenance still needs serviceFluid backlog starts at 4 and follows max(0, 4 + (λ − μ)t). The separate M/M/1 residence-time formula assumes Poisson arrivals, exponential service and λ < μ. Current readings: Fluid backlog: 4; Net rate · λ − μ: −1 /s; M/M/1 mean residence: 1 s. All parameters are adjustable with the labeled controls below.02.557.51001.534.56Elapsed time · sFluid backlogλ < μ
Fluid backlogCurrent time
Fluid backlog
4
Net rate · λ − μ
−1 /s
M/M/1 mean residence
1 s
t = 0 s

Fluid backlog starts at 4 and follows max(0, 4 + (λ − μ)t). The separate M/M/1 residence-time formula assumes Poisson arrivals, exponential service and λ < μ. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forDeferral changes when maintenance is paid; sustained required work still has to fit available service capacity.

Check your understanding

Which measurement supports a meaningful write-amplification comparison?

Try it yourself

An alternative index portfolio produces 60 MB of database-boundary writes for the same 10 MB logical workload. Find its ratio and percentage reduction from the worked example. What evidence is still needed before adopting it?

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

Compare numerator totals at the same boundary, then consider correctness and read performance.

Compare with the solution

The ratio is 60/10=6. Writes fall by (80−60)/80=25% at that stated boundary. Adoption also needs evidence that required constraints remain enforced, relevant reads and ordering remain acceptable, and durability, cleanup completion, workload mix, and measurement conditions are comparable. A lower write ratio alone does not establish a better overall service.

Take it with you

  • Write amplification is a boundary-specific ratio, not a universal hardware constant.
  • MVCC versions, index dependencies, and cleanup determine real maintenance work.
  • Choose an index portfolio from both query benefits and sustained write costs.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.