Cross-encoder reranking score semantics

The model can attend between query and document tokens while producing its score. A cross encoder can therefore distinguish a document that mentions the right entities but states the wrong relationship, although success still depends on training and truncation. The independent document representation used by a dual encoder cannot generally be reused as the complete joint score. Reranking many candidates or long sequences consumes additional inference work, so candidate count, token length, batching, and device conditions determine its practical latency.

The candidate retriever still sets a hard ceiling on what a pure reranker can return. If the relevant document never reaches the candidate set, improving its relative scoring cannot recover it. Measure candidate recall separately from the ordering quality within that set. Also record what text the reranker actually sees: a long document’s decisive passage may be truncated or hidden in a different chunk. A misleading score can arise from missing input evidence rather than an inability to compare two fully observed candidates.

A binary classifier may expose a raw logit or a sigmoid output. Applying a strictly increasing sigmoid preserves its ordering for a fixed query in exact arithmetic, but values between zero and one are not automatically calibrated probabilities on deployment traffic. Training label prevalence, negative sampling, query mix, and domain shift affect interpretation. A pairwise or listwise trained score may have a different meaning altogether. Identify the loss and output transform before using a numeric threshold to reject candidates or route queries.

Calibration asks whether an appropriately defined group of predictions at a given confidence matches its observed relevance frequency. It requires representative labels, an explicit relevance definition, and a held-out procedure. A global calibration curve can hide query-class failures, and a rank-preserving transform can change thresholds while leaving ranking metrics unchanged. The calibration literature establishes that strong classification accuracy does not imply well-calibrated confidence. Do not infer that a score of 0.9 means nine of ten arbitrary documents with that value are relevant without appropriate evidence.

Compare rerankers on identical candidate lists when isolating their ordering effect, then separately evaluate the complete retrieval-and-rerank system. Use held-out queries, check annotation coverage, and report latency under representative batching and sequence lengths. If thresholds control fallback or answer generation, validate those decisions using the relevant costs and class distribution rather than selecting a threshold from one attractive example. Recalibrate or re-evaluate after model, candidate distribution, or label-policy changes; a saved threshold is part of the deployed contract.

The network lab illustrates one logistic scalar, not cross attention. Reset, set input to 1 and weight to 1, predict logit and output, then Step: the fixed bias makes logit 0.5 and output approximately 0.6225. Compare a fresh Reset with both input at 1 and weight at 2 before Step: logit 1.5 gives approximately 0.8176. This unit contains no document tokens, trained reranker, relevance judgments, or calibration experiment.