Bloom filters: rule out membership under an intact contract
Derive the approximate false-positive model, choose a hash count, and preserve the conditions behind no false negatives.
On this page
The question for today
What you’ll understand
- Distinguish a negative membership result from a positive candidate.
- Apply the standard approximate false-positive and optimal-hash formulas with their assumptions.
- Explain why unsafe deletion, corruption, or a mismatched data version can create false negatives.
A Bloom filter spends a compact bit array to avoid some expensive exact lookups. In its ordinary insert-only form, it can say definitely absent or possibly present. The useful guarantee depends on complete insertion and consistent hashing into an intact filter for the correct data set. It is a conditional correctness property, not a promise that an arbitrary stored bit array can never lie.
Set bits conservatively, verify positive answers
Start with m zero bits and k hash positions per key. Inserting a key sets every selected bit to one. A query finding any required bit zero can reject membership because a correctly inserted key would have set it. If all are one, other keys might have set those same bits, so the answer remains a candidate requiring the exact underlying lookup. The filter does not store row values, prove uniqueness, or enumerate matching records. A positive answer can also be a genuine member rather than an error.
The no-false-negative argument assumes the inserted key is queried with the same hashing and encoding rules, every required insertion is reflected, and no relevant bit has been lost or improperly cleared. A filter for yesterday's file cannot safely reject keys added to a different current file unless the design accounts for them. Concurrent construction and publication must preserve that version agreement. Corruption, incomplete writes, incompatible serialization, and stale metadata can violate the premise; a negative result is useful only within the system's actual integrity and lifecycle contract.
Choose space and hash work together
After n insertions, more selected bits are set, so absent queries are harder to reject. Increasing m relative to n lowers saturation, while changing k has a tradeoff: more queried positions initially improve rejection but also set more bits during insertion and cost additional hash work. The familiar continuous optimum is proportional to bits per key times ln two. An implementation chooses a nearby feasible integer and may generate positions using a specified practical hashing scheme. The simple formula is not an exact finite-filter guarantee for every such scheme.
Expected avoided work depends on the query mix. If nearly all queries are real members, the filter adds checking overhead while rejecting little. If most queries are absent and exact lookup is costly, even a modest filter can help. A one-percent false-positive probability among nonmembers is not a one-percent error rate among all queries or positive results. Account for cache misses to the bit array, hashing CPU, and whether multiple underlying files require multiple filter checks. Choose the parameter budget for the workload rather than treating the lowest modeled probability as automatically optimal.
Deletion and rebuilding preserve a versioned set
Clearing the ordinary Bloom bits for a deleted key is unsafe because another live key may depend on those bits. A counting variant can support certain deletion protocols, but must prevent counter underflow, overflow, and deletion of unsupported keys. Rebuilding from an authoritative snapshot can remove obsolete influence if writes arriving during construction are handled and the new filter is published with the correct data version. An immutable storage file naturally supports a file-specific filter; moving or merging its records requires corresponding filter maintenance rather than reusing arbitrary old metadata.
The probability lab models hypothetical false-positive events, not the filter algorithm. Reset, set p=0.01 and n=20, predict a mean of 0.2 events, then Play or Step to inspect exact binomial bars. Separately Reset, set p=0.10 and n=20, predict mean two, then Play or Step. This assumes twenty independent trials with a common stipulated probability. It cannot set m or k, insert keys, or prove no false negatives. Relate the two event rates to avoided exact lookups, then state which filter-integrity premises the bars do not check.
Work it through
Estimate a ten-bit-per-key filter
Use 10,000 filter bits for 1,000 distinct keys under the standard uniform-hashing approximation. Choose k=7 and consider 10,000 absent-key queries.
The space ratio is ten bits per key, and the continuous optimum is 10 ln2≈6.93, making seven a nearby integer choice. The raw bit payload is 10,000/8=1,250 bytes.
Compute the approximate false-positive probability as (1−exp(−7×1,000/10,000))^7≈0.00819, or about 0.819%. It is a nonmember-query probability under the model.
Across 10,000 absent queries, the expected false-positive count is approximately 81.9. Each such positive still performs exact verification; correctly inserted real members must also be handled by the underlying lookup.
The model predicts roughly eighty-two unnecessary exact lookups among ten thousand absent queries, not eighty-two incorrect returned records.
The living notebook
Explore the model
Interactive lab · probability
Count hypothetical false-positive events
Reset, set p=0.01 and n=20, predict mean 0.2 events, then Play or Step. Separately Reset, set p=0.10 and n=20, predict mean 2, then Play or Step. This exact binomial model assumes independent common-probability events; it neither constructs a Bloom filter nor exposes bit-count or hash-count controls.
- Mean · np
- 5
- Variance · np(1−p)
- 2.5
- Total probability
- 1
These are exact binomial probabilities for independent trials with one fixed success probability. Playback tours the distribution; it does not resample outcomes. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.
Look forA false positive adds verification work; integrity and correct versioning are separate prerequisites for safe negative answers.
Pause & reason
Check your understanding
Your turn
Try it yourself
A filter has a modeled nonmember false-positive probability of 0.02. Among 50,000 queries, 40,000 are absent and 10,000 are genuine members. Estimate false positives and explain why 2% of all positive answers is not the right interpretation.
Take a moment to make a prediction or work it out on paper. Open a hint when you need one.
Give me a hint
Apply the probability only to absent queries, then include genuine members among positive results.
Compare with the solution
Expected false positives are 40,000×0.02=800. Under intact insertion and lookup assumptions, all 10,000 genuine members also test positive, so the expected positive-count ratio suggests about 800/10,800≈7.41% false among positives, not 2%. That ratio of expectations is a workload-level approximation, not an exact expectation of a random ratio.
Keep the idea
Take it with you
- A negative answer rules out membership only under intact insertion, hashing, and version agreement.
- False-positive probability is conditioned on an absent key.
- Deletion and rebuilding need explicit invariants rather than arbitrary bit clearing.
Read further
Original sources and deeper explanations to follow at your own pace.
Optional progress, saved in this browser when available. Revisit any lesson at any time.
Lesson not marked complete.