Relational algebra and rewrite rules
Prove query transformations under bag, NULL, and outer-join semantics before considering whether they reduce execution work.
On this page
The question for today
What you’ll understand
- Track duplicate multiplicities through a join and projection.
- Identify the conditions permitting a filter to move below an inner join.
- Construct a counterexample to an invalid outer-join rewrite.
A query optimizer searches among different ways to obtain the required answer. That freedom begins with equivalence, not speed: moving an operator is useful only if the new expression preserves the query's observable semantics. SQL makes the proof richer than textbook set algebra because duplicates, missing values, evaluation behavior, and preserved outer-join rows all affect what an application receives.
SQL normally preserves a bag of rows
A mathematical set records whether an element exists. A bag also records how many times it occurs. Ordinary SQL SELECT does not remove duplicates unless its semantics request duplicate elimination, such as SELECT DISTINCT. Projecting away a column can therefore leave several equal-looking output rows, each of which still matters to a later count or sum. A rewrite that quietly deduplicates an intermediate result may change the answer even when its displayed distinct values look identical. Start a correctness check with multiplicities, not just a list of possible values.
An inner join combines every qualifying pair of input rows. If one key occurs twice on one side and three times on the other, that key contributes six pairs, assuming every associated predicate is true. A declared uniqueness constraint can justify a stronger bound, but a few sample rows cannot establish uniqueness for future data. Keep the needed join keys and multiplicities when pushing projections down. Dropping a column early is harmless only when no later predicate, ordering, grouping, or requested output requires it and the duplicate behavior remains correct.
Selection movement needs an explicit domain
A deterministic total predicate that refers only to the left input can be applied before an inner join or after it, with the same qualifying row pairs. This is a useful selection-pushdown rule. Total here means the expression is defined for every relevant input; otherwise, moving evaluation can change which errors occur. SQL functions with volatile behavior or observable side effects add further restrictions. Optimizers use expression properties and language rules, so an algebraic identity should not be treated as permission to manually reorder arbitrary functions that happen to return booleans.
SQL predicates also use a three-valued logic. An ordinary comparison involving NULL generally produces unknown rather than true or false, and WHERE retains only true rows. NULL is not equal to itself under ordinary equality; use the documented null predicates or null-aware comparison when that is the intended meaning. Rewrites involving NOT IN, equality, and missing values need their own proof. It is often clearer to define the application's missing-value meaning first than to discover after optimization that an empty result came from an unexpected NULL.
An outer join preserves rows for a reason
A left outer join emits unmatched left rows with null-extended right columns. A right-side condition in ON decides whether a match exists; the same condition in WHERE can reject the padded row afterwards. Those are different questions. A null-rejecting right-side WHERE condition can sometimes justify replacing an outer join with an inner join, but preserving the outer join while blindly pushing that condition into its right input is not the same transformation. Write a two-row counterexample before accepting a proposed identity involving a preserved side.
The binomial lab models only the number of rows passing one predicate under an explicit independent equal-probability assumption. Reset, set both p=0.5 and n=20, predict ten expected survivors, then Play or Step. Separately Reset, set both p=0.25 and n=20, predict five, then Play or Step. Playback highlights exact outcomes rather than sampling rows. This instrument has no joins, NULLs, duplicate identities, or rewrite engine. Use it to estimate a potential work reduction only after proving the selection movement legal; it cannot establish equivalence.
Work it through
ON and WHERE disagree on a preserved customer
Customers A and B each occur once. Orders contain one A row with amount 40 and no B rows. Compare a left join with amount>50 inside ON against a left join followed by WHERE amount>50.
With the amount condition in ON, A has no qualifying order and B has no order at all. The left join preserves both customers, producing two rows with null-extended order columns.
With only customer equality in ON, the intermediate output has the A order for 40 and one padded B row. The subsequent WHERE condition is false for A and unknown for B.
WHERE retains only true predicates, so the second query returns zero rows. The first returns two. Changing placement has therefore changed semantics before any cost or execution advantage is considered.
Preserved rows and unknown predicates make this a counterexample to unrestricted filter movement across a left outer join.
The living notebook
Explore the model
Interactive lab · probability
Estimate survivors only after proving equivalence
Reset, set both p=0.5 and n=20, predict ten expected successes, then Play or Step. Separately Reset, set both p=0.25 and n=20, predict five, then Play or Step. Exact binomial bars assume independent common-p row survival; they neither sample a table nor prove a rewrite correct.
- Mean · np
- 5
- Variance · np(1−p)
- 2.5
- Total probability
- 1
These are exact binomial probabilities for independent trials with one fixed success probability. Playback tours the distribution; it does not resample outcomes. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.
Look forReducing expected rows is a performance opportunity only when the transformed query preserves its required answer.
Pause & reason
Check your understanding
Your turn
Try it yourself
A bag has projected values red, red, blue. What are COUNT(*) and the number of rows after SELECT DISTINCT? Why can an early DISTINCT change a later sum?
Take a moment to make a prediction or work it out on paper. Open a hint when you need one.
Give me a hint
Distinguish occurrences from distinct values before applying an aggregate.
Compare with the solution
COUNT(*) is three, while DISTINCT produces two rows. A later sum or count ordinarily includes every retained occurrence, so deleting a duplicate can reduce an aggregate even though no new distinct value disappears. Any rewrite introducing duplicate elimination needs a semantic reason, such as a proven key or an explicitly duplicate-insensitive consumer.
Keep the idea
Take it with you
- Prove SQL rewrites with duplicates and NULL behavior included.
- Selection pushdown across an inner join needs suitable expression properties.
- Outer-join preservation can make ON and WHERE placements inequivalent.
Read further
Original sources and deeper explanations to follow at your own pace.
Optional progress, saved in this browser when available. Revisit any lesson at any time.
Lesson not marked complete.