advanced29 min studyLesson 41 in Engineering

Data models and access paths

Choose keys, constraints, and indexes from concrete query shapes and measured selectivity rather than database fashion.

On this page

What you’ll understand

  • Translate an access pattern into a key ordering and bounded scan.
  • Estimate page work without equating selectivity with elapsed time.
  • Balance read improvements against index maintenance and data invariants.

A model-serving dashboard asks for one tenant's latest successful training runs. The database contains every tenant, every state, and years of history. A useful data model must represent which facts are valid, while an access path determines how much work finding those facts requires. These are connected decisions: a perfectly indexed collection of ambiguous identifiers can return the wrong answer very quickly. Begin with a precise query, its ordering, and the invariant its caller expects.

Give identities and relationships an explicit meaning

Suppose each run belongs to exactly one tenant, has a stable run identifier, and moves through a defined state machine. A composite uniqueness rule on tenant and run identifier prevents duplicate local identities. A foreign key can protect a relationship to an existing tenant; application validation alone races with concurrent changes. Separate an immutable creation time from a mutable completion time because sorting by one answers a different question from sorting by the other.

Choose normalization around ownership of facts. Duplicating a mutable tenant name into every run makes historical display easy but creates an update policy. Is the field a deliberate historical snapshot or an accidentally stale copy? A materialized read model can be valuable when its source, freshness bound, and rebuild procedure are clear. Denormalization removes joins from one read path while moving consistency work into writes or asynchronous projection maintenance.

Nmatch=sN,s=NmatchNN_{\mathrm{match}}=sN,\qquad s=\frac{N_{\mathrm{match}}}{N}
Selectivity s is the matching fraction of N rows for a specified predicate. Correlated predicates cannot generally be estimated by multiplying their individual selectivities.↔ Scroll wide equations horizontally.

Order an index around the question

For tenant equality followed by newest completion time, a B-tree beginning with tenant and then completion time can bound the tenant range and traverse it in the desired order. A state predicate changes the design: placing state before time helps an equality-constrained state query but changes ordering across states. A partial index for successful rows may fit a stable workload. Inspect the actual predicate and parameter behavior before assuming the planner can use it.

An index is a second maintained structure. Inserts and updates pay for its pages, logging, and cache footprint. Included payload columns can reduce table visits in suitable index-only scans, but visibility information and engine behavior still matter. PostgreSQL can sometimes use skip scans on later columns; therefore avoid the simplistic claim that a missing first-column predicate makes a composite index unusable. Ask how much of the structure is visited and why.

Measure the work a plan actually performs

Use representative data and fresh statistics, then compare estimated and observed row counts with an execution plan. A sequential scan may win for a large matching fraction or a tiny cached table. Distinguish index entries examined from rows returned, and logical buffer hits from physical reads. Run repeated measurements under the expected concurrency; a fast isolated query can displace useful pages or amplify random reads when many tenants arrive together.

A page-count estimate is a useful starting bound, not a latency prediction. Assume fixed-size rows and ignore page headers, fragmentation, and compression for the calculation below. Real queries add tree navigation, row visibility checks, sorting, and possibly scattered table access. Preserve a baseline plan and test the slow tenant as well as the median tenant. Optimize an access path only after verifying that its output still obeys the required consistency and ordering rules.

Pscan≈NbB,Pidealrange≈sNbBP_{\mathrm{scan}}\approx\frac{Nb}{B},\qquad P_{\mathrm{ideal range}}\approx\frac{sNb}{B}
N is rows, b is bytes per row, and B is usable bytes per page. The ideal contiguous range estimate excludes index traversal and random table fetches; it is not a general B-tree cost formula.↔ Scroll wide equations horizontally.

Bound a tenant's latest-run query

A table has 8 million 128-byte rows and assumed 8,192 usable bytes per page. One tenant owns 0.25% of rows. Compare a full scan with an ideal contiguous tenant range before applying a LIMIT.

  1. Estimate total bytes as 8,000,000 × 128 = 1,024,000,000, giving 125,000 idealized pages.

  2. The tenant owns 0.0025 × 8,000,000 = 20,000 rows, or 2,560,000 bytes.

  3. Its ideal contiguous range occupies 312.5 pages, so round up to 313 pages before traversal and table-fetch overhead.

  4. A matching ordered index may stop after the requested recent rows, but filtering unsuccessful rows can require extra entries. Confirm that work in the actual plan.

The result

The range is far smaller than the table, but its 313-page idealization does not guarantee a particular response time.

Explore the model

Interactive lab

Interactive lab · probability

Inspect a toy selectivity distribution

Set p as an assumed independent match probability and n as sampled rows. The lab shows exact binomial bars, mean np, and variance np(1−p), without resampling. Real tenant membership and clustered pages can be correlated; this is a sampling illustration, not a query optimizer.

Inspect a toy selectivity distributionThese are exact binomial probabilities for independent trials with one fixed success probability. Playback tours the distribution; it does not resample outcomes. Current readings: Mean · np: 5; Variance · np(1−p): 2.5; Total probability: 1. All parameters are adjustable with the labeled controls below.03581000.070.150.220.3Successes kP(X = k)k = 0 · P = 9.77e-4
Exact probabilityHighlighted outcomeMean np
Mean · np
5
Variance · np(1−p)
2.5
Total probability
1
t = 0 s

These are exact binomial probabilities for independent trials with one fixed success probability. Playback tours the distribution; it does not resample outcomes. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forMatching fraction and uncertainty describe rows, while physical organization determines the access work.

Check your understanding

Why might a database correctly prefer a sequential scan despite an available index?

Try it yourself

A 2 million-row table has 200-byte rows and a query matches 1%. With 8,000 usable bytes per page, estimate full-scan and ideal matching-range pages. Name one reason an actual index plan could read more.

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

Compute bytes before pages, then apply selectivity only to the ideal contiguous matching range.

Compare with the solution

The full table is 400 million bytes, or 50,000 idealized pages. One percent is 20,000 rows and 4 million bytes, giving 500 range pages. Index navigation, scattered heap fetches, visibility checks, or reading extra entries before filtering can increase actual work. Compression and caching also change the relationship between pages and latency.

Take it with you

  • Model identity and relationship invariants before optimizing lookup speed.
  • Match index ordering to concrete equality, range, and ordering requirements.
  • Use representative execution plans to validate simple page-work estimates.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.