NVIDIA Publishes Quantization Recipe Compressing 321B GLM-5.3-Flash to Fit a Single 288GB GPU
The IQ2_S plus NVFP4 recipe reports an average accuracy delta of just -0.1 points versus FP8, halving serving hardware needs.
NVIDIA's Model Optimizer team published post-training quantization recipes for GLM-5.3-Flash, a 321-billion-parameter mixture-of-experts model with 304B parameters across 288 routed experts. A full IQ2_S recipe at 2.56 bits per weight produces a roughly 130GB model but loses 2.3 points on average across six benchmarks versus FP8. The recommended hybrid applies NVFP4 only to expert down-projections, at an effective 3.21 bits per weight, producing about 155GB with an average delta of just -0.1 plus or minus 0.3 points, effectively matching FP8 without any retraining.
The team's analysis found the majority of quantization error concentrates in expert down-projections, so moving only that tensor class cut NLL error by roughly 40%. Quantization-Aware Distillation fixed long-context runaway behaviour but did not recover knowledge recall, so the official recommendation is the pure PTQ recipe. FP8 serving requires two 288GB GPUs, while the quantised model fits one. The results reshape on-premise cost estimates for frontier-class MoE serving.