Welcome to Warpspace Blog
We are the creators of Caveduck.io - Your AI Companion Service. Follow our journey in AI, technology, and innovation.
That Free Korean NPU the Government Hands Out, Can You Actually Use It? Notes From Porting an Unsupported Model
Moving a persona-deviation detector (an mmBERT-based classifier) that we served on GPU in production onto a Korean NPU (Rebellions ATOM+): the journey from recognizing the problem to writing the adapter, compiling on real hardware, and benchmarking.
Neural Network Compilation, Made Simple: How a PyTorch Model Becomes Hardware Language
What exactly does it mean to ‘compile a model’? From the difference with eager execution to graph capture, operator fusion, kernel lowering, static shapes, and memory planning: we walk through it step by step with diagrams and an interactive demo.
Efficient ML #5, QAT: Undoing Quantization Error During Training
PTQ works for big models but collapses on small models and low bits (down to 13.6% at 2-bit). The fix is QAT: folding quantization into training itself. We insert fake quantization into the forward pass and punch through round’s zero-gradient problem with the STE (Straight-Through Estimator). If round has zero derivative, how does anything learn? Answered with plots measured directly on an MNIST MLP.
Efficient ML #4, Low-bit Quantization: Fighting Outliers
FP4 has only 15 representable values, so how do modern GPUs run inference in 4 bits? The culprit that breaks low-bit quantization is the outlier. From an experiment where a single outlier erases an entire layer, to four cures, group quantization (MXFP4), KL clipping, AdaRound, and Hadamard rotation, we verify each in code, including an experiment where 2-bit accuracy comes back from 53% to 97%.
Efficient ML #3: Neural Net Pruning
Starting from LeCun’s 1989 Optimal Brain Damage, this is the story of shrinking a model by cutting weights away (pruning). What to cut, how, and how much; the surprise of compression that survives even a 95% cut thanks to retraining; and the point that you only get a real payoff when the hardware (NVIDIA 2:4) backs you up, all with plots we measured ourselves on an MNIST MLP.
Efficient ML #2: Neural Net Quantization
We take the trained 32-bit weights of a neural net and shrink them down to 2–8 bits using two approaches: K-Means (non-uniform) and Linear (uniform, integer arithmetic). From the affine mapping r=S(q−Z) all the way to the compression-ratio vs. accuracy trade-off, with plots measured by actually running the code on an MNIST MLP.
Efficient ML #1: Deep Learning Data Types
INT8, FP16, BF16, FP8 (E4M3/E5M2), FP4: what do all these data types flooding deep learning model cards actually mean, and how do bits get interpreted as numbers? We take them apart one by one, with a widget where clicking bits updates the formula and value in real time.
Welcome to Warpspace Blog
Introducing the official Warpspace Blog - where we share our journey building Caveduck.io