Moving a persona-deviation detector (an mmBERT-based classifier) that we served on GPU in production onto a Korean NPU (Rebellions ATOM+) — the journey from recognizing the problem to writing the adapter, compiling on real hardware, and benchmarking.
What exactly does it mean to ‘compile a model’? From the difference with eager execution to graph capture, operator fusion, kernel lowering, static shapes, and memory planning — we walk through it step by step with diagrams and an interactive demo.
Starting from LeCun’s 1989 Optimal Brain Damage, this is the story of shrinking a model by cutting weights away (pruning). What to cut, how, and how much; the surprise of compression that survives even a 95% cut thanks to retraining; and the point that you only get a real payoff when the hardware (NVIDIA 2:4) backs you up — all with plots we measured ourselves on an MNIST MLP.
We take the trained 32-bit weights of a neural net and shrink them down to 2–8 bits using two approaches: K-Means (non-uniform) and Linear (uniform, integer arithmetic). From the affine mapping r=S(q−Z) all the way to the compression-ratio vs. accuracy trade-off, with plots measured by actually running the code on an MNIST MLP.
INT8, FP16, BF16, FP8 (E4M3/E5M2), FP4 — what do all these data types flooding deep learning model cards actually mean, and how do bits get interpreted as numbers? We take them apart one by one, with a widget where clicking bits updates the formula and value in real time.
Introducing the official Warpspace Blog - where we share our journey building Caveduck.io