A Compressed 4-Bit Model Beat Its Own Full-Precision Original

Shrinking a large language model almost always comes at a cost in quality. The standard recipe for efficient deployment goes like this: first you trim the architecture, dropping layers, attention heads or neurons, then you quantize the remaining weights down to 4 bits to cut memory and compute requirements. Both steps save resources, but together they hit precisely the abilities the models are used for in the first place: reasoning, math and code generation. That's why serious pipelines add a recovery stage known as “healing.”

In their paper on Quantization-Aware Healing (QAH), researchers at Multiverse Computing tackled a question that had remained open until now: how do you properly recover a model that has undergone not only quantization but also structural compression? Applying QAH to GPT-OSS 120B, compressed down to 60B parameters and quantized to the MXFP4 format, they arrived at a result that upends the usual hierarchy: the 4-bit model beat its own full-precision (bfloat16) version on 7 of 9 benchmarks — while being smaller, cheaper to run, and more accurate than the very checkpoint it was derived from.

The catch with previous methods

Healing is usually done through fine-tuning with fake quantization (QAT) or through distillation from a frozen full-precision teacher (QAD). But after structural compression, an independent full-precision version of the reduced architecture simply doesn't exist. The only available teacher is the recovered bfloat16 checkpoint itself, which is already an approximation of the original. Distilling from it ties the student to an already degraded target and caps it at that checkpoint's ceiling.

The QAH idea

The authors change one detail: they distill directly from the original, pre-compression model. Teacher and student don't even share an architecture — the teacher is full-size and full-precision, the student is half the size and runs in MXFP4. Because the distribution of the teacher's outputs doesn't depend on architecture, the size mismatch doesn't get in the way of knowledge transfer. Quantization thus stops being a “post-processing loss” and turns into a second distillation pass — the student picks up what the recovery stage simply didn't manage to convey.

Results

The biggest gains came precisely in the abilities that compression usually damages the most: long-context reasoning (+7.4 on AA-LCR) and math (+5.6 on AIME 2025). On two benchmarks (MMLU-Pro and SciCode) the QAH model comes up short, but by less than a point and a half. What's more, with half the parameters and roughly a quarter of the memory for weights, it outperforms the original 120B teacher on LiveCodeBench.

A separate advantage is stability. In a head-to-head comparison on a 9B model, both methods reach a close peak (54.9 versus 54.6), but QAH gets there in about 100 steps — seven times faster than QAT — and barely slips afterward. QAT, by contrast, degrades sharply after its peak, losing about 19 points by step 1200, which means it requires careful early stopping.

The authors' conclusion: a compressed 4-bit model isn't doomed to be a worse copy of a full-precision one. With the right healing recipe it can be smaller, cheaper and more accurate all at once, and quantization turns from an “efficiency tax” into an extra opportunity to train the model.

Source: Hugging Face Blog