Merge shard answers under an explicit exactness contract

Assume shards partition the eligible logical documents without overlap. All use the same deterministic total order: first the exact query score, then a global stable-ID tie key. If a document lies below position k on its own shard, that shard alone contains at least k documents ahead of it, so it cannot appear in the global top k. Therefore the union of exact local top k sets contains every global winner, and an exact coordinator sort can select the answer.

The shared total order matters at ties. If each shard breaks equal distances differently from the coordinator, a globally preferred tied ID might be omitted locally. Score compatibility matters too: independently trained compressed codebooks can introduce different approximation biases. Comparing their reported approximate distances does not restore the original exact order. Exact reranking on common original vectors can make candidate scores comparable, but the local candidate sets must still contain the global winners.

A replica holds another copy of a shard’s documents to improve availability or serving capacity. It does not add new eligible IDs. Select a replica for each shard or deduplicate by logical ID and version before the final ranking. Overlapping partitions can also merge correctly when each local top-k ranks unique logical IDs under that same total order. Trouble appears when multiple physical versions or duplicate entries consume slots within a local result before deduplication: top-k over entries is then not top-k over logical documents. A larger local candidate count alone is a heuristic.

A missing shard changes the answer domain. Returning the best items from responding shards may be a useful degraded response, but it cannot be labeled exact global top-k without additional information proving the absent shard contains no better item. Record coverage explicitly. A replica’s stale snapshot can also resurrect old versions or miss recent documents, so shard coverage and snapshot coverage must both be checked before treating a merged result as complete.

With local ANN search, the coordinator can compute an exact top k of the received candidates while still missing the corpus’s true top k. Distinguish those statements in metrics and API documentation. Oversampling local candidates and exact reranking can improve recall, but there is no universal multiplier that repairs every graph miss or unprobed IVF cell. Benchmark the entire fan-out, filtering, deduplication, and merge path against a frozen exact reference.

Latency is also a distributed property: waiting for all required shards makes the slowest necessary response relevant, while speculative replica requests add resource cost. Measure the result deadline, timeout policy, coverage fraction, and coordinator work, rather than summing isolated single-shard timings. Keep shard count fixed when isolating local search parameters; changing partitioning alters both candidate distributions and network fan-out, so it is a different system configuration that needs fresh evaluation.