Hash indexes: equality still needs collision handling
Separate key equality, hash equality, bucket mapping, overflow behavior, and workload assumptions behind expected lookup costs.
On this page
The question for today
What you’ll understand
- Explain why sharing a bucket or hash value does not prove key equality.
- Calculate average bucket load under an explicit uniform-hashing model.
- Compare equality access with range ordering and skew-sensitive overflow costs.
A hash index maps a key to a bucket so an equality search can avoid an ordered tree descent. The mapping deliberately sends many possible keys into fewer storage locations. Collisions are therefore an ordinary part of the design, not an exceptional proof that hashing failed. A correct lookup must recover the exact key semantics after using the hash to narrow its candidates.
Keep three equality questions separate
Two keys can map to the same bucket while having different full hash values, and two distinct keys can also share a full finite-width hash. Neither condition implies the original values are equal. A generic chained hash table can store keys alongside entries and compare them while searching its bucket. A database index may instead retain only a hash fingerprint and a row reference, requiring the table value for exact rechecking. PostgreSQL 18's hash index stores a four-byte hash value rather than the original indexed datum.
The hash function must respect the database's equality semantics: values considered equal must be mapped consistently for the supported operation. Bytewise hashing without accounting for collation or a type's equality rules can violate that requirement. Conversely, a good hash is not a total order. The neighboring bucket of a key does not correspond to the next larger key, so a hash equality index does not directly provide the ordered range navigation of a B-tree. Evaluate the operator support of the actual index type rather than its name alone.
Expected constant work needs assumptions
For N stored entries distributed uniformly over B buckets, the expected load per bucket is N/B. That average is not a bound on the largest bucket and does not describe a workload with many identical values. Chaining, open addressing, overflow pages, and resizing have different costs. A highly duplicated key can concentrate many required matches into one bucket even with an excellent hash function. Returning those matches necessarily costs work proportional to the output, so a constant-time headline cannot mean constant time for every full result.
Persistent database hash indexes also need crash recovery, concurrent bucket splits, and safe page reclamation. PostgreSQL's implementation can add overflow pages to a full bucket and split buckets as it expands. A lookup may inspect overflow pages, making a skewed bucket more expensive than a shallow B-tree path. Deleting tuples and recycling pages does not necessarily shrink the index file or bucket count. Rebuild behavior and maintenance policy are implementation details to measure; do not assume a standard in-memory hash-table resize routine describes an on-disk database.
Inspect skew before choosing the access path
Compare equality queries with realistic popular keys, absent keys, and a changing table size. Record candidate rechecks, bucket or overflow page visits, cache behavior, and insertion latency during growth. A small average load can coexist with one expensive hot bucket. If a query also requires sorting, range conditions, uniqueness enforcement, or multiple search columns, confirm the access method supports them; PostgreSQL 18's hash indexes have narrower capabilities than its B-tree indexes. Space saved by fingerprints must be weighed against exact recheck work.
The modular lab exposes one deliberately weak bucket map. Reset, set modulus m=16 and additive step a=4, predict the orbit 0,4,8,12 repeating with period four, then Play or Step. Separately Reset, set m=16 and a=5, predict all sixteen residues appear, then Play or Step. Interpret the sequence as stride-patterned integer keys under a direct modulo map only. It is not a production hash function, randomized collision experiment, or database bucket-split simulation. The contrast demonstrates why input structure and mixing assumptions matter.
Work it through
Collide without confusing the keys
A teaching hash table uses b(x)=x mod 16 and stores keys 5,21,37. Separately consider 64 entries assigned independently and uniformly to sixteen buckets.
The three explicit keys all map to bucket 5 because each differs from 5 by a multiple of 16. They remain three distinct keys, so a lookup for 21 must perform the table's exact-key comparison.
For the independent uniform model, expected load is 64/16=4 entries per bucket. This expectation says nothing about the exact occupancy of bucket 5 in the explicit three-key example.
There are 64×63/2=2,016 unordered pairs in the uniform model. Each shares a bucket with probability 1/16, giving 126 expected colliding pairs; this is not 126 distinct crowded buckets.
The explicit collision and the random-model expectation answer different questions. Neither permits returning every bucket entry as an exact match.
The living notebook
Explore the model
Interactive lab · modular
A stride can occupy only part of the bucket space
Reset, set m=16 and a=4, predict period 4 with residues 0,4,8,12, then Play or Step. Separately Reset, set m=16 and a=5, predict period 16, then Play or Step. This additive modulo sequence is a weak-map illustration, not randomized hashing or a persistent hash index.
- Residue r0
- 0
- Orbit period · m/gcd(m,a)
- 12
- Distinct residues visited
- 1 of 12
Addition modulo m starts at zero: rₙ = na mod m. Playback advances two additions per second. The orbit has period m/gcd(m,a), including period one for a = 0. This is not a cryptographic security test or a primality algorithm. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.
Look forA uniform bucket assumption requires evidence about the mapping and input distribution.
Pause & reason
Check your understanding
Your turn
Try it yourself
A uniform model stores 2,000 entries in 500 buckets. Find expected load per bucket. Explain why a workload with 1,000 identical indexed values violates a naive uniform-entry placement model.
Take a moment to make a prediction or work it out on paper. Open a hint when you need one.
Give me a hint
Equal values must map consistently rather than independently choose new buckets.
Compare with the solution
Expected load under independent uniform choices is 2,000/500=4. But 1,000 identical values share the same hash and relevant bucket mapping, so their placements are not independent uniform draws across buckets. Returning all of those matches already requires output-proportional work, and their concentration can create overflow or long chains.
Keep the idea
Take it with you
- Hash equality and bucket equality do not establish original key equality.
- Expected access costs depend on load, distribution, output size, and collision resolution.
- Hash access trades ordered navigation for an equality-oriented mapping with its own maintenance.
Read further
Original sources and deeper explanations to follow at your own pace.
Optional progress, saved in this browser when available. Revisit any lesson at any time.
Lesson not marked complete.