Vector indexes and retrieval

An embedding is a coordinate representation produced by a particular model and preprocessing pipeline. Mixing incompatible model versions in one index can make distances meaningless even if dimensions match. Cosine similarity divides the dot product by both vector norms; Euclidean distance measures coordinate displacement. For nonzero unit-normalized vectors, maximizing cosine similarity is equivalent to minimizing squared Euclidean distance. Without that normalization assumption, norm differences can change the ranking substantially.

An exact scan computes the chosen distance against every eligible vector and selects the best k. It supplies a reference neighbor set for index accuracy, with a documented tie policy. Estimate its arithmetic and memory traffic before dismissing it: a small filtered tenant subset may be cheap enough for exact search. Conversely, a large unfiltered corpus can be dominated by reading vectors even when each distance calculation is simple. Quantization changes numerical representation and introduces another approximation to evaluate.

Graph-based HNSW search navigates a layered proximity graph rather than scanning all vectors. Its search breadth and construction choices affect recall, memory, build cost, and latency. There is no general guarantee that the first approximate answer is the exact nearest point. Other families partition the space or compress vectors, spending different resources. Compare configurations at matched accuracy and workload.

Define recall at k against exact top-k neighbors for the same metric and eligible corpus. That geometric recall is not user relevance, answer faithfulness, or permission correctness. Measure those downstream outcomes separately with appropriate labels. Stratify by rare topics, new documents, and challenging filters rather than reporting one mean. A retrieval system can improve average recall while missing a small tenant's important documents, especially if the benchmark mostly reflects the largest and easiest populations.

Filtering after an approximate candidate search can leave fewer than k eligible results, even when many eligible documents exist elsewhere. Increasing search effort, iterating the scan, prefiltering with a suitable plan, or partitioning the index may help depending on the engine. Authorization must still be enforced on returned documents; an embedding similarity score never grants access. Evaluate the actual query plan and parameter settings because merely creating a vector index does not ensure every query uses it.

A production index must represent inserts, updates, and deletions within a declared freshness bound. Record embedding version, source revision, deletion status, and rebuild generation. Account for vector bytes and graph overhead as well as database tuples and build scratch space. During a model upgrade, compare old and new retrieval quality in isolated generations and establish how readers switch. Test update-heavy operation and post-filter tail latency, not only a static build benchmark with a warm cache.