https://blog.caveduck.io/tags/%EC%96%91%EC%9E%90%ED%99%94/Recent content in 양자화 on Warpspace BlogHugo -- 0.154.5en-USMon, 11 May 2026 00:00:00 +0000https://blog.caveduck.io/posts/quantization-aware-training/Mon, 11 May 2026 00:00:00 +0000https://blog.caveduck.io/posts/quantization-aware-training/PTQ works for big models but collapses on small models and low bits (down to 13.6% at 2-bit). The fix is QAT: folding quantization into training itself. We insert fake quantization into the forward pass and punch through round’s zero-gradient problem with the STE (Straight-Through Estimator). If round has zero derivative, how does anything learn? Answered with plots measured directly on an MNIST MLP.https://blog.caveduck.io/posts/low-bit-quantization/Mon, 04 May 2026 00:00:00 +0000https://blog.caveduck.io/posts/low-bit-quantization/FP4 has only 15 representable values, so how do modern GPUs run inference in 4 bits? The culprit that breaks low-bit quantization is the outlier. From an experiment where a single outlier erases an entire layer, to four cures, group quantization (MXFP4), KL clipping, AdaRound, and Hadamard rotation, we verify each in code, including an experiment where 2-bit accuracy comes back from 53% to 97%.