Merge updates, not arbitrary factor sums

Under column-vector convention, standard LoRA computes W₀x+sB(Ax) with s=alpha/r. In exact arithmetic and without dropout or other active adapter behavior, this equals(W₀+sBA)x. The same frozen base and target orientation must be used for both paths. Training-time adapter dropout must be disabled for the simple deterministic equivalence. Additional mechanisms or biases can change the full transformation and need their documented merge semantics.

Floating-point operation order can make the separate and merged paths differ numerically even when the exact-real expressions match. Quantized bases add dequantization and requantization choices;merging into low-bit storage may approximate the update. Repeated merge/unmerge cycles need not restore the original quantized bytes. Preserve an immutable base and adapter artifact rather than using repeated inverse operations as the only rollback mechanism. Test the exact inference dtype and backend used in release.

For two updates s₁B₁A₁ and s₂B₂A₂, their sum can be represented exactly by concatenating appropriately scaled B factors horizontally and stacking A factors vertically. The combined factor width can be r₁+r₂. Compressing it back to a smaller rank is generally an approximation unless additional structure makes the sum lower rank. Quality after that compression must be evaluated rather than inferred from the original adapters' separate scores.

Adding B₁+B₂ and A₁+A₂ creates cross terms B₁A₂ and B₂A₁, so it does not generally equal the desired sum of updates. PEFT 0.17.0 documents multiple merging methods with different assumptions and restrictions;identify the selected method rather than treating a weighted-adapter API name as one universal algebra. Also verify base compatibility and target sets. Two adapters trained from different base revisions cannot automatically be added as if their deltas shared the same reference point.

Compare separate-adapter and merged outputs on representative fixed inputs under the same decoding and numeric settings. Include cases near decision boundaries and long context where small logit differences can change generated text. Evaluate each task and retention set after composition because adding updates is not a linear interpolation of model behavior. Record merge weights, algorithm, dtype, quantization configuration, and immutable source hashes. A successful tensor-shape load does not establish the intended base or output equivalence.

The matrix lab is a restricted orientation aid. Reset and set both a=1, b=1, predict its determinant 1 and shear, then Step. It shows only [[a, b], [0,1]], not arbitrary adapter factors or a rank-compression algorithm. Readouts describe the chosen matrix while geometry interpolates. Use it to distinguish an input vector from an output mapping, but calculate the BA example below independently;the sliders cannot express every merged update or forecast its language-model quality.