core33 min studyLesson 238 in Mathematics

Randomized algorithms and expected cost

Keep inputs fixed while averaging internal choices, analyze randomized quicksort with indicators, and distinguish expected cost from error guarantees.

On this page

What you’ll understand

  • Specify the probability space in a randomized runtime claim.
  • Derive the pairwise comparison probability for randomized quicksort.
  • Distinguish Las Vegas correctness, Monte Carlo error, and tail bounds.

Randomization can protect an algorithm from a fixed unfavorable ordering by making its decisions unpredictable to that ordering. The guarantee is not that every random execution is fast. Instead, a cost variable is averaged over the algorithm's random bits while the input remains fixed. This distinction lets us prove a uniform expected bound for every input, without pretending real data arrive from a convenient distribution. It also keeps runtime uncertainty separate from the possibility of returning a wrong answer.

What is random, and what must always be correct?

A Las Vegas algorithm always returns the correct answer whenever it terminates, with random running time; a stated finite expected runtime also implies termination with probability one in the usual nonnegative cost model. A Monte Carlo algorithm may return an incorrect answer with a specified bounded probability, often within a fixed time budget. These labels are descriptions of guarantees, not of particular programming syntax. Randomized quicksort with ordinary correct partitioning is Las Vegas: every pivot sequence produces a valid sorted order.

Assume distinct keys and choose each recursive pivot uniformly from that subarray. For each fixed input, the random choices determine the comparison count. A poor sequence can repeatedly choose an extreme key and take quadratic time, but the expected count is O(n log n). The quantifier order matters: each fixed input receives a good expectation over fresh choices. An adaptive adversary that can observe and influence future pivots requires a separate model; the fixed-input proof does not automatically cover it.

C=∑1≤i<j≤nXij,E[C]=∑i<jPr⁡(Xij=1)C=\sum_{1\le i<j\le n}X_{ij},\qquad \mathbb E[C]=\sum_{i<j}\Pr(X_{ij}=1)
Xᵢⱼ indicates whether the keys of sorted ranks i and j are compared. Linearity of expectation does not require these indicators to be independent.↔ Scroll wide equations horizontally.

Count comparisons by the first separating pivot

Two ranked keys i and j are compared exactly when one of them is the first pivot selected among ranks i through j. If an interior key is selected first, it separates them into different subproblems before they can meet. If an endpoint is first, its partition step compares the pair. Uniform pivot selection makes every rank in that interval equally likely to be the first selected, so the probability is two divided by the interval's number of ranks.

Summing these probabilities yields at most a constant times n times the harmonic sum, which grows logarithmically. More precisely, the count is 2(n+1)Hₙ−4n for the standard partition model comparing a pivot once with each other key in its subarray. This is an expectation of comparisons, not a universal exact runtime: swaps, recursion management, duplicate handling, and memory behavior require their own accounting. Distinct keys avoid a duplicate-partitioning ambiguity rather than being an irrelevant cosmetic assumption.

Pr⁡(Xij=1)=2j−i+1,E[C]=2(n+1)Hn−4n=O(nlog⁡n)\Pr(X_{ij}=1)=\frac{2}{j-i+1},\qquad \mathbb E[C]=2(n+1)H_n-4n=O(n\log n)
Hₙ is the sum of reciprocal integers from one through n. The formula assumes distinct keys, uniform recursive pivots, and the stated pivot-comparison counting convention.↔ Scroll wide equations horizontally.

Expectations are not high-probability guarantees

A nonnegative cost variable satisfies Markov's inequality: the probability it exceeds t times its mean is at most one divided by t for positive t. That bound is useful but weak. An expected logarithmic factor does not by itself establish concentration around the mean or a tiny failure probability. Stronger quicksort tail analyses use additional structure. Similarly, an unbiased estimator can occasionally be far from its expectation; controlling variance or a tail probability supplies information that unbiasedness alone lacks.

Independent repetition can reduce a Monte Carlo error only when the decision rule and error type justify it. If a one-sided test misses a witness with probability at most one half per independent run, accepting any found witness leaves a miss probability at most two to the minus number of runs. Reusing correlated randomness invalidates that multiplication. In implementation, pseudorandom generators approximate the algorithmic source of randomness under their intended assumptions; neither an animation nor a single benchmark establishes the formal distributional guarantee.

Average all pivot choices on three keys

For three distinct keys, calculate the expected number of pivot comparisons in randomized quicksort. Count a pivot compared once with each other key in its current subarray.

  1. The first partition always makes two comparisons. With probability one third the pivot is the middle key, leaving only singleton children and no more comparisons.

  2. With probability two thirds the first pivot is an extreme key. A two-key subproblem remains and makes exactly one further comparison, regardless of which of those two keys becomes its pivot.

  3. The expectation is 2 + (2/3)·1 = 8/3. The pair-indicator check gives probabilities one, one, and two thirds for the two adjacent-rank pairs and the extreme pair, with the same sum.

The result

The expected count is 8/3, while any one run uses either two or three comparisons. An expected cost need not be an integer or equal a possible execution cost.

Explore the model

Interactive lab

Interactive lab · sampling

Separate mean behavior from a distribution

Pause and reset, then set both controls to Bernoulli p = 0.5 and sample size n = 4. Predict sample-mean variance 0.0625 and inspect the exact bars. For a fresh comparison, pause and reset, then set p = 0.5 and n = 16; predict variance 0.015625 and a narrower distribution. Playback highlights exact outcomes rather than drawing new samples. This independent Bernoulli model illustrates expectation and spread; it is not quicksort's correlated comparison distribution or a proof of its tail bound.

Separate mean behavior from a distributionExact binomial masses for the mean of n independent Bernoulli trials, compared with normal mass between adjacent half-count boundaries; normal tails outside the finite support remain outside the bars. Playback highlights outcomes, not random samples. The Berry–Esseen bound applies to standardized CDFs, not individual bars or continuity-corrected errors. At p = 0 or 1, normal comparison and that bound are disabled. Normal-CDF evaluation uses a numerical approximation with absolute error below 7.5×10⁻⁸ (at most 1.5×10⁻⁷ per interval, before rounding). Current readings: Mean sample proportion: 0.5; Sample-mean variance: 0.0125; Uniform CDF error bound: ≤ 0.1062. All parameters are adjustable with the labeled controls below.Exact distribution of the mean of 20 independent Bernoulli variables with success probability 0.5; playback highlights outcome 0.00.250.50.75100.0580.1150.1730.23Sample mean · k / nProbability massNormal bin masses are an approximationk = 0 · exact probability = 9.54e-7
Exact binomial massNormal interval massHighlighted outcome / mean
Mean sample proportion
0.5
Sample-mean variance
0.0125
Uniform CDF error bound
≤ 0.1062
t = 0 s

Exact binomial masses for the mean of n independent Bernoulli trials, compared with normal mass between adjacent half-count boundaries; normal tails outside the finite support remain outside the bars. Playback highlights outcomes, not random samples. The Berry–Esseen bound applies to standardized CDFs, not individual bars or continuity-corrected errors. At p = 0 or 1, normal comparison and that bound are disabled. Normal-CDF evaluation uses a numerical approximation with absolute error below 7.5×10⁻⁸ (at most 1.5×10⁻⁷ per interval, before rounding). Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forA mean does not specify every execution; variance and tails answer different questions.

Check your understanding

Why may quicksort's pair-indicator expectations be added even though the indicators are dependent?

Try it yourself

A nonnegative randomized cost has expectation at most 120 steps on each fixed input. What does Markov's inequality guarantee about cost at least 600? Separately, a one-sided test misses with probability at most 1/4 per independent run. Bound the miss probability after three runs when any success is accepted.

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

Use the threshold directly for Markov; use the event that all three tests miss for repetition.

Compare with the solution

Markov gives probability of cost at least 600 at most 120/600 = 1/5. This is a tail bound, not a promise that cost is always below 600. For three independent one-sided runs, all must miss for the combined test to miss, so the probability is at most (1/4)³ = 1/64. The second result uses independence and the stated acceptance rule; neither follows merely from an expected runtime bound.

Take it with you

  • Randomized guarantees specify which input is fixed and which internal choices are averaged.
  • Pair indicators yield quicksort's expected comparison bound without requiring independence.
  • Always-correct outputs, expected resource use, and small error probabilities are separate guarantees.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.