Note, , 1 min read
What Survives Compression
Semantic retention as a way of thinking about evaluation
Ask what meaning survived, not how fluent the remainder sounds.
Contents
Summary
Compression is a useful metaphor for what generative systems do to source. A long context becomes a short answer. A document becomes a schema. A policy becomes a refusal. The question that matters is not whether the remainder is elegant. It is what survived.
At IJCNN 2025, my co-authors and I introduced SrCr — Semantic Retention Compression Rate — a measure of what remains after compression. The paper is about models. The habit it recommends is broader: score the retained meaning before you score the style of the remainder.
That habit is also how I like to work: measurement before fluency, and the name on a report is the name on the work.
What SrCr is for
It is easy to compress a text and keep a sentence that sounds like the original. It is harder to keep the claim, the constraint, the exception. SrCr asks how much of the meaning survived the squeeze, not how pretty the squeeze looks.
The same question applies when a RAG system answers from a pile of pages, or an extractor fills a schema from a scan. The remainder can be fluent and still be a different document.
Measurement before fluency
Fluency is cheap now. It is the wrong north star for a system that touches money, identity, or medical fact. The evaluation has to ask whether the claims are still the claims, whether the absences are still absences, whether the refusal is still the policy.
A judge that rewards well-written error will pass a system that should have been held. Calibrate the judge against that failure, or do not use a judge.
From the paper to a run
In practice the analogue is simple. Name the meaning that must survive — the fields, the citations, the policy clauses. Measure retention on those, on the real distribution. Write the line. The rest of the report is how the number was made.
The paper remains the longer argument. This note is only the habit it left in my work.