Choose the geometry before choosing the index

Squared Euclidean distance adds the squared coordinate differences and ranks smaller values first. Faiss L2 indexes report this squared quantity, so a returned value of nine corresponds to Euclidean distance three. Taking a square root preserves ranking but changes thresholds and units. Maximum inner product instead ranks larger dot products first. A positive rescaling of one fixed query rescales every inner product equally; independently rescaling stored vectors can change which document wins.

Cosine similarity divides the dot product by both norms, so it compares direction rather than magnitude. That removes information when an embedding model deliberately uses norm to express confidence, popularity, or another learned signal. Treat normalization as part of the model contract rather than an automatic improvement. Record whether the query encoder and document encoder are symmetric or deliberately different, and check their documented scoring rule before selecting the index metric.

For nonzero vectors, dividing by the Euclidean norm places them on the unit sphere. Expanding the squared difference of two unit vectors leaves two minus twice their dot product. Thus descending cosine, descending normalized inner product, and ascending normalized squared L2 produce the same order in exact arithmetic. This statement requires normalization on both sides. Normalizing only the query leaves database norms in the inner-product score and does not establish the equivalence.

A zero vector has no direction, so its cosine is undefined. Adding a tiny denominator does not discover a missing direction; it defines a new scoring convention. A production ingestion policy might reject zero embeddings, quarantine them for re-encoding, or assign a separately specified fallback route. Apply the same policy during evaluation. Also reject nonfinite coordinates and check normalization tolerances, because NaNs and overflow can poison both training and sorting before the search algorithm begins.

A test oracle should compute the chosen score against the same eligible document snapshot as the approximate index. Specify whether distance is squared, how ties are broken, and whether document IDs represent unique logical items. A deterministic secondary key such as ascending ID makes exact top-k sets reproducible when scores tie. Floating-point implementations may disagree near a boundary, so retain distances and inspect a tolerance-aware tie analysis alongside strict ID intersection.

Try separating geometric quality from semantic usefulness. Exact neighbors under a poorly trained encoder can still be irrelevant to a person. Conversely, a semantically useful result can count as a geometric miss. Use an exact search to isolate index approximation, then a separate task evaluation to judge the embedding model. When a new encoder changes its dimension or normalization, rebuild the reference scores before comparing indexes; a familiar array shape alone does not preserve the old geometry.