Randomized algorithms and expected cost
Keep inputs fixed while averaging internal choices, analyze randomized quicksort with indicators, and distinguish expected cost from error guarantees.
On this page
The question for today
What you’ll understand
- Specify the probability space in a randomized runtime claim.
- Derive the pairwise comparison probability for randomized quicksort.
- Distinguish Las Vegas correctness, Monte Carlo error, and tail bounds.
Randomization can protect an algorithm from a fixed unfavorable ordering by making its decisions unpredictable to that ordering. The guarantee is not that every random execution is fast. Instead, a cost variable is averaged over the algorithm's random bits while the input remains fixed. This distinction lets us prove a uniform expected bound for every input, without pretending real data arrive from a convenient distribution. It also keeps runtime uncertainty separate from the possibility of returning a wrong answer.
What is random, and what must always be correct?
A Las Vegas algorithm always returns the correct answer whenever it terminates, with random running time; a stated finite expected runtime also implies termination with probability one in the usual nonnegative cost model. A Monte Carlo algorithm may return an incorrect answer with a specified bounded probability, often within a fixed time budget. These labels are descriptions of guarantees, not of particular programming syntax. Randomized quicksort with ordinary correct partitioning is Las Vegas: every pivot sequence produces a valid sorted order.
Assume distinct keys and choose each recursive pivot uniformly from that subarray. For each fixed input, the random choices determine the comparison count. A poor sequence can repeatedly choose an extreme key and take quadratic time, but the expected count is O(n log n). The quantifier order matters: each fixed input receives a good expectation over fresh choices. An adaptive adversary that can observe and influence future pivots requires a separate model; the fixed-input proof does not automatically cover it.
Count comparisons by the first separating pivot
Two ranked keys i and j are compared exactly when one of them is the first pivot selected among ranks i through j. If an interior key is selected first, it separates them into different subproblems before they can meet. If an endpoint is first, its partition step compares the pair. Uniform pivot selection makes every rank in that interval equally likely to be the first selected, so the probability is two divided by the interval's number of ranks.
Summing these probabilities yields at most a constant times n times the harmonic sum, which grows logarithmically. More precisely, the count is 2(n+1)Hₙ−4n for the standard partition model comparing a pivot once with each other key in its subarray. This is an expectation of comparisons, not a universal exact runtime: swaps, recursion management, duplicate handling, and memory behavior require their own accounting. Distinct keys avoid a duplicate-partitioning ambiguity rather than being an irrelevant cosmetic assumption.
Expectations are not high-probability guarantees
A nonnegative cost variable satisfies Markov's inequality: the probability it exceeds t times its mean is at most one divided by t for positive t. That bound is useful but weak. An expected logarithmic factor does not by itself establish concentration around the mean or a tiny failure probability. Stronger quicksort tail analyses use additional structure. Similarly, an unbiased estimator can occasionally be far from its expectation; controlling variance or a tail probability supplies information that unbiasedness alone lacks.
Independent repetition can reduce a Monte Carlo error only when the decision rule and error type justify it. If a one-sided test misses a witness with probability at most one half per independent run, accepting any found witness leaves a miss probability at most two to the minus number of runs. Reusing correlated randomness invalidates that multiplication. In implementation, pseudorandom generators approximate the algorithmic source of randomness under their intended assumptions; neither an animation nor a single benchmark establishes the formal distributional guarantee.
Work it through
Average all pivot choices on three keys
For three distinct keys, calculate the expected number of pivot comparisons in randomized quicksort. Count a pivot compared once with each other key in its current subarray.
The first partition always makes two comparisons. With probability one third the pivot is the middle key, leaving only singleton children and no more comparisons.
With probability two thirds the first pivot is an extreme key. A two-key subproblem remains and makes exactly one further comparison, regardless of which of those two keys becomes its pivot.
The expectation is 2 + (2/3)·1 = 8/3. The pair-indicator check gives probabilities one, one, and two thirds for the two adjacent-rank pairs and the extreme pair, with the same sum.
The expected count is 8/3, while any one run uses either two or three comparisons. An expected cost need not be an integer or equal a possible execution cost.
The living notebook
Explore the model
Interactive lab · sampling
Separate mean behavior from a distribution
Pause and reset, then set both controls to Bernoulli p = 0.5 and sample size n = 4. Predict sample-mean variance 0.0625 and inspect the exact bars. For a fresh comparison, pause and reset, then set p = 0.5 and n = 16; predict variance 0.015625 and a narrower distribution. Playback highlights exact outcomes rather than drawing new samples. This independent Bernoulli model illustrates expectation and spread; it is not quicksort's correlated comparison distribution or a proof of its tail bound.
- Mean sample proportion
- 0.5
- Sample-mean variance
- 0.0125
- Uniform CDF error bound
- ≤ 0.1062
Exact binomial masses for the mean of n independent Bernoulli trials, compared with normal mass between adjacent half-count boundaries; normal tails outside the finite support remain outside the bars. Playback highlights outcomes, not random samples. The Berry–Esseen bound applies to standardized CDFs, not individual bars or continuity-corrected errors. At p = 0 or 1, normal comparison and that bound are disabled. Normal-CDF evaluation uses a numerical approximation with absolute error below 7.5×10⁻⁸ (at most 1.5×10⁻⁷ per interval, before rounding). Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.
Look forA mean does not specify every execution; variance and tails answer different questions.
Pause & reason
Check your understanding
Your turn
Try it yourself
A nonnegative randomized cost has expectation at most 120 steps on each fixed input. What does Markov's inequality guarantee about cost at least 600? Separately, a one-sided test misses with probability at most 1/4 per independent run. Bound the miss probability after three runs when any success is accepted.
Take a moment to make a prediction or work it out on paper. Open a hint when you need one.
Give me a hint
Use the threshold directly for Markov; use the event that all three tests miss for repetition.
Compare with the solution
Markov gives probability of cost at least 600 at most 120/600 = 1/5. This is a tail bound, not a promise that cost is always below 600. For three independent one-sided runs, all must miss for the combined test to miss, so the probability is at most (1/4)³ = 1/64. The second result uses independence and the stated acceptance rule; neither follows merely from an expected runtime bound.
Keep the idea
Take it with you
- Randomized guarantees specify which input is fixed and which internal choices are averaged.
- Pair indicators yield quicksort's expected comparison bound without requiring independence.
- Always-correct outputs, expected resource use, and small error probabilities are separate guarantees.
Read further
Original sources and deeper explanations to follow at your own pace.
Optional progress, saved in this browser when available. Revisit any lesson at any time.
Lesson not marked complete.