Bitmap access: combine candidate sets, then recheck what was lost
Distinguish persistent bitmap structures from transient bitmap scans, trace AND/OR combinations, and account for lossy page-level candidates.
On this page
The question for today
What you’ll understand
- Compute intersections and unions of candidate row-location sets.
- Explain why a bitmap heap scan loses index ordering.
- Distinguish exact tuple locations, lossy page candidates, predicate rechecks, and MVCC visibility.
A bitmap lets an engine combine candidate locations from multiple searches before visiting table data. That can turn scattered accesses into a page-oriented visit order. It does not mean every database stores a permanent bitmap index, and it does not mean every bit proves a visible matching row. Identify what a bit represents and what information the plan must recheck.
Separate the index type from the scan strategy
A persistent bitmap index is a stored representation whose design may associate bit vectors with values or ranges, often with compression. A PostgreSQL Bitmap Index Scan instead builds a temporary in-memory candidate-location structure from an ordinary supported index access path. Several such results can be combined by BitmapAnd or BitmapOr, then consumed by a Bitmap Heap Scan. Calling that plan a persistent bitmap index conflates the search structure with the execution strategy. Their write behavior and storage requirements can be very different.
For an exact row-location model, an AND keeps only locations reported by every contributing predicate; an OR keeps locations reported by at least one. Duplicate locations in an OR are visited once in the set representation. Candidate row identities must refer to compatible table storage and snapshot processing, not arbitrary identifiers from unrelated tables. A combined candidate still needs the normal visibility and any required predicate checks. Set algebra organizes where to look; it does not create transaction semantics or make an approximate access method exact.
Lossy location information requires rechecking
Under memory pressure, a transient bitmap can represent a whole heap page rather than every matching tuple offset on that page. This lossy entry means that relevant candidates may be somewhere on the page, not that every row satisfies the original condition. The executor must inspect the page's tuples and recheck the necessary predicates. Access methods can also return lossy candidates for other reasons. Do not infer that a visible Recheck Cond means every page was lossy; examine exact and lossy page statistics and the access method's contract.
MVCC visibility is a separate check from predicate truth. A row location can pass the indexed condition but reference a version the current query cannot see. An exact tuple bitmap does not remove visibility processing, and a lossy bitmap does not authorize returning false matches. Conversely, lossy representation must preserve a superset of required candidates so matching rows are not omitted. When evaluating memory settings, compare added recheck work and page visits against memory consumption rather than treating a smaller bitmap as automatically faster.
Page order trades away index order
Visiting heap pages in physical order can improve locality, but it discards the original B-tree result order. A query requiring ORDER BY may therefore need a separate sort, and a small LIMIT may favor an ordered index scan that stops early. Combining two selective indexes also requires performing both searches and building the bitmap. A planner can rationally choose one index plus filtering, a composite index, or a sequential scan instead. The useful choice depends on candidate density, page distribution, cache state, and sorting costs.
The cache lab isolates page reuse rather than building a bitmap. Reset, set C=4 and s=4, predict twelve hits in the initial sixteen requests, then Play or Step. Separately Reset, set C=4 and s=1, predict zero hits because the sixteen-address cycle exceeds capacity, then Play or Step. The model cannot reorder a real candidate list, expose lossy tuple bits, or perform rechecks. Use it to explain why the mapping from rows to physical pages matters, while doing the actual candidate intersections on paper.
Work it through
Combine exact candidate locations
One index yields candidate row locations A={1,3,4,8}; another yields B={3,4,6}. Assume these are exact locations in one table and handle visibility separately.
For predicate A AND B, intersect the sets to obtain {3,4}, containing two candidate locations.
For predicate A OR B, take {1,3,4,6,8}, containing five locations. The count also follows 4+3−2=5, so overlapping locations are not double-counted.
If a lossy representation expands one candidate to its whole page, examine additional tuples and recheck predicates before returning visible matches. Neither the original set order nor SQL ORDER BY is preserved automatically.
The exact AND has two candidates and the OR has five. These counts concern locations before any remaining visibility or predicate processing.
The living notebook
Explore the model
Interactive lab · cache
Location density and reuse are separate from set algebra
Reset, set C=4 and s=4, predict 12 initial hits out of 16 requests, then Play or Step. Separately Reset, set C=4 and s=1, predict zero hits, then Play or Step. This LRU address stream does not combine bitmaps, reorder heap visits, or implement predicate rechecks.
- Hits / requests
- 0 / 16
- Hit rate
- 0%
- Address-stream period
- 16 addresses
A fully associative LRU cache starts empty and requests address (i·s) mod 16. Hits move an address to the newest position; misses evict the oldest when full. Playback starts with 16 requests and adds two per second, capped at 4096 explicitly simulated requests. This toy stream is not a processor-cache benchmark. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.
Look forCandidate-set correctness and the locality of resulting page visits require different reasoning.
Pause & reason
Check your understanding
Your turn
Try it yourself
Two exact candidate sets have 120 and 80 locations, with 30 in common. Find AND and OR sizes. If a lossy candidate page has 40 visible rows but only three satisfy the condition, how many of those rows may be returned?
Take a moment to make a prediction or work it out on paper. Open a hint when you need one.
Give me a hint
Apply inclusion-exclusion, then distinguish visibility from predicate truth.
Compare with the solution
The AND contains 30 locations, and the OR contains 120+80−30=170. On the lossy page, only the three visible rows satisfying the condition may be returned for that predicate. Forty visible rows do not imply forty matches; the other thirty-seven must be rejected by rechecking.
Keep the idea
Take it with you
- Transient bitmap scans are distinct from stored bitmap index formats.
- Lossiness broadens candidates and requires rechecking without omitting true matches.
- Page-oriented visits improve some access patterns while losing index ordering.
Read further
Original sources and deeper explanations to follow at your own pace.
Optional progress, saved in this browser when available. Revisit any lesson at any time.
Lesson not marked complete.