Mixed-Precision Quantization

Not every part of a neural network is equally comfortable with being reduced to low-precision numbers. Mixed-precision quantization treats the model like a budget: use very compact arithmetic where it is safe, and spend extra precision only where the model genuinely needs it.

How it works
Quantization replaces floating-point values, such as 32-bit FP32 weights and activations, with smaller formats such as INT8, INT16, or FP16. In mixed precision, different operations, layers, tensors, or even channels use different formats. A convolution layer whose output barely changes after rounding can run in INT8, while a sensitive layer—perhaps the first input layer, a depthwise convolution, or the final classifier—keeps INT16 or FP16. The goal is not to use the lowest precision everywhere; it is to find the lowest precision that preserves acceptable accuracy.

Choosing where precision belongs
Engineers measure how much each part of a trained model contributes to prediction error after quantization. A practical workflow is:

Quantize the whole model to INT8 as a baseline.
Test accuracy and inspect layers that create the largest error.
Restore those sensitive operations to a higher precision.
Benchmark the resulting model on the actual target hardware.

This can be done after training with calibration data, or more reliably through quantization-aware training, where the model learns while simulating rounding effects. The runtime must support the chosen mix: TensorFlow Lite, for example, can use INT8 paths and selected higher-precision operations, while mobile NPUs may impose their own supported formats.

Why it matters on devices
A wake-word detector on a small battery-powered device might run most layers in INT8 to reduce flash use, RAM traffic, and energy, while retaining INT16 activations in one accuracy-critical stage. A smart camera can similarly fit object detection inside its thermal budget without losing small or dim objects. Blindly forcing every layer to INT8 can reduce recognition quality; retaining FP32 everywhere wastes memory and power. Mixed-precision quantization makes that trade-off deliberate, matching the model to the device rather than asking the device to carry an unnecessarily expensive model.

Mixed-precision quantization assigns different numerical precisions—such as INT8, INT4, or FP16—to different model layers, channels, weights, or activations according to their sensitivity to reduced precision. Critical operations retain higher precision while tolerant ones use fewer bits. This reduces memory use, bandwidth, energy, and inference latency more aggressively than uniform quantization while preserving accuracy on constrained edge hardware.

Imagine packing for a trip with limited suitcase space. You keep delicate items in sturdy boxes, but save space by folding everyday clothes tightly. Mixed-precision quantization does something similar for an AI model on a phone, camera, or sensor.

AI models store lots of numbers. Some parts need more detail to stay accurate, while others can use simpler, smaller numbers without much effect. Mixed precision lets the device use the right amount of detail in each part.

This helps the model take up less memory, use less battery, and often respond faster—while protecting the parts that matter most for good results.