Teacher Model

A teacher model is the capable, usually larger model used to guide the training of a smaller model. Think of it as an expert tutor: it does not need to fit on the final device, but it helps a compact “student” learn more than it could from labels alone.

How the teacher teaches
In ordinary training, an image labeled “cat” gives a model one hard answer: cat. A teacher also supplies its full pattern of confidence: perhaps 92% cat, 6% fox, and 2% dog. These are called soft predictions or soft targets. They reveal useful relationships between classes that the single correct label hides.

The teacher is trained first, or is an already strong existing model.
For each training example, it produces output scores called logits.
A small student model learns from both the true label and the teacher’s outputs.
A temperature setting can soften the teacher’s probability distribution, making its secondary preferences easier for the student to learn.

Why this matters on devices
The teacher can be too large, slow, and power-hungry for deployment—perhaps a cloud-scale vision network needing gigabytes of memory. The student can be small enough for TensorFlow Lite on a phone, an NPU-equipped camera, or a Cortex-M microcontroller, while retaining much of the teacher’s accuracy. For example, a large speech model can teach a compact wake-word detector to distinguish “Hey device” from similar background speech without running the large model continuously on a battery-powered product.

A training tool, not the shipped model
The teacher normally runs only during training infrastructure, where compute is available. The student is the model packaged into the edge application. Without a good teacher, a very small model trained only on labels can lose accuracy sharply; with distillation, it gains a richer learning signal without increasing its deployed memory, latency, thermal load, or battery draw.

A teacher model is a larger, higher-accuracy model whose predictions or internal representations supervise a smaller student model during knowledge distillation. The teacher transfers learned behavior through softened output probabilities, feature targets, or both. For edge AI, it enables compact student models to retain much of the teacher’s accuracy while meeting device limits on memory, compute, power, and latency.

Think of a teacher model as an expert tutor helping a smaller student learn the same job. The teacher is a large, powerful AI that has learned from lots of examples and can make very accurate judgments. But it may be too big, slow, or power-hungry to run on a phone, camera, or sensor.

Instead of putting the teacher directly on the device, developers use it to train a smaller “student” AI. The student learns to copy the teacher’s useful decisions, then runs on the device itself. This lets everyday gadgets use capable AI while saving battery, memory, and processing power.