Reciprocal-rank fusion and score combination

Use stable document identities before combining lists, especially when one branch returns chunks and another returns whole documents. Deduplication and document aggregation can change ranks and must be defined. For the practical truncated-list convention used here, an absent document contributes zero from that branch. It is not assigned an imaginary rank just below the cutoff. Increasing one branch’s depth can therefore change a fused score even when its top results are unchanged. Report branch cutoffs together with the fusion parameter.

Ranks begin at one in the displayed formula. A larger positive k reduces the relative advantage of the top few positions; the original paper fixed k at 60 after a pilot study. That choice is an experimental configuration, not a universal optimum. Ties need a documented policy, such as deterministic ordering before assigning ordinal ranks or a declared shared-rank scheme. Missing candidates and duplicate lists also matter: copying the same list twice supplies extra weight, not independent new relevance evidence.

A linear combination of BM25 and dense scores can be dominated by whichever feature has the larger numeric range, even if that feature is less useful. Specify transforms, missing-feature handling, and weights fitted on held-out queries. A per-query min-max transform can be sensitive to one outlier and undefined for a constant list without a fallback; z-score normalization also requires a meaningful spread estimate. Normalization changes feature scale but does not prove probability calibration. Evaluate it as part of the ranking system.

Rank fusion is invariant to strictly increasing transforms that preserve each list’s order, while a weighted raw-score sum usually is not. That robustness costs information: a narrow margin and a huge margin receive the same rank contribution. Choose based on observed query classes, candidate recall, latency, and labels rather than a belief that one fusion family is inherently safe. Correlated retrievers can reinforce the same mistakes, so a second branch’s existence alone does not establish useful complementarity.

Start with held-out queries and log every branch rank, raw score, transformed score, and final decision for inspected errors. Compare each branch alone, their candidate union, and the final ordering to locate whether an improvement comes from coverage or ranking. Freeze the judgment policy and audit novel unjudged documents. If a learned fusion model uses query features, split evaluation by query and time so the model cannot memorize related documents or benefit from labels that would not exist in deployment.

The vector lab provides coefficients for a separate hand calculation only. Reset, set angle to 0 degrees and magnitude to 1, predict components, then Step: the coefficient vector is (1,0). Compare a fresh Reset with both angle at 90 degrees and magnitude at 1 before Step: (0,1). For fixed features A = (0.9,0.2) and B = (0.3,0.8), compute their dot products by hand and observe the winner changes. The instrument does not fuse ranked lists or learn weights.