Context compression and summary loss

Token deletion preserves selected original symbols but can remove a negation, unit, reference target, or dependency needed to interpret them. An abstractive summary can synthesize statements and introduce unsupported details; a learned state may not be directly readable or compatible with another model. Record the compressor, tokenizer, prompt, model version, and target length. A compression ratio says how many tokens remain under that tokenizer, not how many facts or constraints survive. A concise fluent result can still omit the one exception the downstream query requires.

Query-aware compression can retain the evidence needed for a known question, but a reusable summary must support questions not known when it was created. Those objectives differ. If two source records with different required answers map to the same summary, a downstream function seeing only that summary cannot distinguish them. More capable generation can add prior knowledge, yet that does not reconstruct a missing fresh value reliably. Keep the original material available when exact recovery or new questions are part of the system contract.

Attach source identities, versions, and supporting spans to retained claims so later answers can verify them against the original. A summary citation to an entire document may be too coarse to support a precise date or exception. Distinguish quoted evidence from generated inference, retain uncertainty, and avoid converting a source’s tentative statement into a categorical one. When sources disagree, collapsing them into one clean sentence can destroy both the disagreement and its provenance. A downstream citation should resolve to the actual source, not merely repeat the summary’s authority.

Evaluate omissions and additions separately. Missing a necessary fact can cause failure without any hallucinated text, while a confident unsupported addition can poison later answers. Construct paired sources differing only in a task-relevant detail and test whether compression preserves that distinction. Also vary question types: a summary good for broad themes may fail exact lookup, numeric comparisons, or multi-document reconciliation. Recursive summaries can compound loss across levels, so inspect both intermediate and final representations rather than validating only the last fluent paragraph.

Include the compressor’s inference cost, source retrieval, provenance storage, and downstream generation in the resource comparison. A smaller final prompt can still cost more end to end if compression is expensive or repeated for every query. Cache reusable summaries only under a versioned source and question-scope contract, and invalidate them when the source changes. Report task quality alongside token reduction, including exact constraints and citations, so a favorable average does not hide loss of rare but important details.

The probability lab illustrates only an assumed independent retention model. Reset, set per-detail retention probability to 0.9 and detail count to 10, predict the probability all survive, then Step: approximately 0.3487. Compare a fresh Reset with both probability at 0.9 and count at 20 before Step: approximately 0.1216. Real compression errors can be correlated and the rate is not known from this instrument. It does not summarize text, assess provenance, or establish an actual retention probability.