LiteRT
LiteRT is the small, device-focused engine that lets a trained machine-learning model do useful work without sending every input to a cloud server. It is built for the realities of phones, cameras, embedded Linux boards, and tiny battery-powered devices: limited memory, limited energy, and hardware that cannot simply be upgraded when a model is too large.
What LiteRT does
Formerly known as TensorFlow Lite, LiteRT is Google’s on-device inference runtime and deployment toolkit. A model is converted into a compact .tflite FlatBuffer model containing its network structure and learned weights. On the device, the LiteRT interpreter reads that model, allocates working memory, and executes its operators—such as convolutions, matrix multiplications, and activations—to turn inputs into predictions.
How it reaches device hardware
LiteRT does not have to run every calculation on the main CPU. It can use delegates: components that hand supported parts of a model to faster hardware, including a phone GPU or an NPU (neural processing unit). This is less like replacing the whole engine and more like assigning suitable tasks to specialist workers.
- On Android, a delegate can accelerate image classification or pose estimation.
- On a Cortex-M microcontroller, LiteRT for Microcontrollers runs a very small, carefully chosen operator set with a fixed memory arena.
- For a wake-word detector, an int8-quantized model can listen continuously while using far less memory and power than floating-point inference.
Why it matters at the edge
Choosing LiteRT affects model design well before deployment: the model’s operators must be supported, its weights must fit storage, and its temporary tensors must fit RAM. Quantization—representing values with smaller integer types—is central because it shrinks models and can unlock efficient accelerator execution. Ignoring these constraints can leave a model that is accurate in a notebook but too slow, too hot, or simply impossible to load on the target device. LiteRT makes private, low-latency inference practical even when the network is unavailable.
LiteRT is Google’s on-device inference runtime and deployment stack, renamed from TensorFlow Lite, for running trained AI models efficiently on Android, iOS, embedded Linux, web, and other edge targets. It executes compact models through CPU, GPU, NPU, and accelerator backends using application-facing APIs. At the edge, LiteRT enables low-latency, private, offline inference within fixed device memory, power, and hardware constraints.
Think of LiteRT as a compact playback app for AI. Just as a music player lets your phone play a song without sending it to a distant studio, LiteRT lets a device run an AI model right where it is.
It is Google’s lightweight on-device AI runtime, previously known as TensorFlow Lite. A phone camera might use it to recognize objects, or a smart sensor might spot unusual sounds. Because the AI can run locally, it can respond quickly, keep more data private, and still work when there is no internet connection.