advanced27 min studyLesson 630 in Engineering

Release the whole adapter dependency tuple

Bind adapter weights to their base, tokenizer, targets, and serving configuration, and verify a real rollback path.

On this page

What you’ll understand

  • Identify every artifact needed to reconstruct an adapted model.
  • Validate the final serving and merge/quantization path rather than only training weights.
  • Design rollback without relying on numerically reversible unmerging.

An adapter file is a delta and configuration interpreted against another model, not a self-contained application. A successful load can still use the wrong base revision, tokenizer, template, target set, or numeric path. Release the complete dependency tuple and evaluate the exact served artifact. Rollback should restore known immutable sources and routing state rather than assume that disabling one component reverses every training or merge change.

Compatibility is more than tensor shape

Record immutable base weights/configuration, adapter tensors and configuration, target module names, rank/scaling, tokenizer files, chat template, added tokens, and relevant library/runtime versions. PEFT 0.17.0 documents an adapter checkpoint as adapter state plus configuration rather than a complete frozen base. A base with identical dimensions but different weights changes the adapter's reference point. A tokenizer with the same vocabulary count can still change IDs and stopping behavior, so shape compatibility is only one validation layer.

If training also changes embeddings, biases, or other saved modules, include them in the release and rollback contract. Quantization settings and compute dtype belong with the final inference path. A merged full checkpoint is a different artifact from a base-plus-adapter pair and may have irreversible rounding changes. Keep source hashes and the exact merge recipe so provenance and reconstruction do not depend on mutable repository names or an operator remembering which adapter was active.

R=(Hbase,Hadapter,Htokenizer,Htemplate,Ctargets,Cruntime)\mathcal R=(H_{\rm base},H_{\rm adapter},H_{\rm tokenizer},H_{\rm template},C_{\rm targets},C_{\rm runtime})
R is a release manifest tuple;H denotes content identities or immutable revisions and C explicit configuration. It is a provenance structure, not a cryptographic proof that training data or model behavior is safe.↔ Scroll wide equations horizontally.

Evaluate the path that users will call

Test fixed prompts, token IDs, stop conditions, task metrics, retention, severe failures, and resource/latency behavior after loading the actual release. Compare separate-adapter and merged execution under the intended dtype and quantization, accepting only documented numerical differences. Canary traffic needs compatible cohorts and a defined stop condition. A training checkpoint's validation score does not cover a deployment that silently uses another template or truncation policy;those changes need their own evidence.

Shared serving adds state isolation. Routing must select the intended adapter and base for each request, and any prefix or key/value cache reuse must satisfy the implementation's model-state compatibility rules. Cache entries produced under one adapter are not automatically valid under another simply because prompt text matches. Include model/adapter revisions and other required state in cache identity or invalidate appropriately. Test concurrent tenants and in-flight requests during switches rather than checking only a quiet single-request load.

KV⁡θ1(P)≢KV⁡θ2(P)in general\operatorname{KV}_{\theta_1}(P)\not\equiv\operatorname{KV}_{\theta_2}(P)\quad\text{in general}
For prompt P and differing adapted model parameters theta, the resulting attention-cache state is not generally interchangeable. Some specialized reuse may be valid under a documented implementation contract, but prompt equality alone does not establish it.↔ Scroll wide equations horizontally.

Rollback restores known state and drains old work

A rollback can route new requests to an immutable prior release while allowing or terminating old in-flight work under an explicit policy. Clear or segregate incompatible caches and preserve the old tuple long enough for controlled recovery. Disabling a pure adapter returns the intended base path only if no other trained modules or artifacts changed. After a quantized merge, loading the original base is more reliable than assuming an inverse subtraction reproduces its original bytes.

The cache lab is only a version-identity contrast. Reset and set both C=4, s=4, predict twelve hits among sixteen requests, then Step to add one hit. Its empty-start fully associative LRU stream caps at 4096 requests and has no model weights, KV state, tenants, or release routing. Use the contrast to explain that a cache hit is correct only under a valid identity contract;this toy's matching address cannot certify that a real prompt cache belongs to the newly selected adapter.

A small adapter still needs a base

A release targets two ordinary dense matrices, each with 1024 inputs and 1024 outputs, using rank four and no extra trained modules. Adapter weights use two bytes per parameter. The base and tokenizer are separate pinned artifacts.

  1. Each target has 4×(1024+1024)=8192 adapter parameters. Two targets contain 16,384 parameters total.

  2. Adapter weight payload is 16,384×2=32,768 bytes, or 32 KiB, before file headers and configuration. That small payload does not contain the frozen base weights, tokenizer, or chat template.

  3. A rollback must restore the intended earlier manifest and compatible caches/routing, not just replace this 32 KiB file while leaving a mismatched base or template. Verify output and retention on the reconstructed tuple before considering the release reproducible.

The result

Artifact size is not deployment completeness. The adapter's dependencies and stateful serving path are part of the release.

Explore the model

Interactive lab

Interactive lab · cache

Release identity controls safe reuse

Reset, set C=4, s=4;predict 12 hits in 16 requests, then Step adds one hit. This capped 4096-request address-cache toy has no adapters, KV tensors, model hashes, or tenant routing.

Release identity controls safe reuseA fully associative LRU cache starts empty and requests address (i·s) mod 16. Hits move an address to the newest position; misses evict the oldest when full. Playback starts with 16 requests and adds two per second, capped at 4096 explicitly simulated requests. This toy stream is not a processor-cache benchmark. Current readings: Hits / requests: 0 / 16; Hit rate: 0%; Address-stream period: 16 addresses. All parameters are adjustable with the labeled controls below.Fully associative LRU cache after 16 deterministic requests: 0 hits and 16 misses.Most recent 16 requestsaddress = i × 3 mod 160M3M6M9M12M15M2M5M8M11M14M1M4M7M10M13MCurrent cache order · oldest → newest471013Latest address 13 · miss: inserted as newestToy address stream · period 16 · starts empty
Cached addressMiss / evictionMost recent request
Hits / requests
0 / 16
Hit rate
0%
Address-stream period
16 addresses
t = 0 s

A fully associative LRU cache starts empty and requests address (i·s) mod 16. Hits move an address to the newest position; misses evict the oldest when full. Playback starts with 16 requests and adds two per second, capped at 4096 explicitly simulated requests. This toy stream is not a processor-cache benchmark. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forServing caches and rollback must preserve the complete model-state compatibility contract.

Check your understanding

An adapter loads into a same-shaped but different base revision. What is justified?

Try it yourself

A quantized merged model is being rolled back to its original base. Explain why subtracting an adapter update from the merged low-bit weights is not a reliable byte-exact rollback, and name the immutable artifacts to preserve instead.

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

Quantization and repeated merging can lose information that subtraction cannot reconstruct.

Compare with the solution

Rounding or requantization can discard information, so an inverse numerical operation need not recover the original base bytes. Preserve the original immutable base, adapter/configuration, tokenizer/template, runtime/quantization settings, and prior release manifest. Restore the verified tuple and segregate incompatible caches and in-flight routing according to policy. Confirm actual outputs and retention rather than assuming a mathematically inverse merge restored the deployed state.

Take it with you

  • An adapter release includes immutable dependencies and final serving configuration.
  • Merge, quantization, tokenization, and cache routing can change deployed behavior.
  • Rollback should restore a verified tuple rather than rely on reversible arithmetic.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.