Mean Intersection over Union (mIoU)

Imagine placing a predicted colored mask on top of the correct mask for an image and asking: how much do they truly cover the same pixels? Mean Intersection over Union (mIoU) turns that visual comparison into a single, reliable score for evaluating segmentation models.

How the score is calculated
For one class, such as “road,” Intersection over Union (IoU) compares the pixels predicted as road with the ground-truth road pixels:

IoU = correctly shared pixels / pixels in either mask

The numerator is the intersection: pixels both the model and annotation label as that class. The denominator is the union: every pixel labelled as that class by either one. An IoU of 1 means perfect agreement; 0 means no overlap. mIoU calculates IoU separately for every class, then takes their average. For example, a street-scene model might score road, car, pedestrian, building, and sky independently before averaging them.

Why averaging by class matters

Pixel accuracy can look excellent when a model labels abundant background pixels correctly while missing rare but important objects.
mIoU gives each class equal influence, so poor pedestrian or defect segmentation cannot be hidden by a large, easy-to-segment background.
False positives and false negatives both lower IoU: predicting extra pixels outside an object enlarges the union, while missing object pixels reduces the intersection.

Use in real vision systems
mIoU is a standard benchmark metric for semantic segmentation in autonomous driving, medical scans, satellite imagery, and factory inspection. A model segmenting tumors, for instance, needs strong overlap with the expert annotation—not merely a high count of correct background pixels. Libraries such as TorchMetrics provide multiclass Jaccard/IoU metrics to compute these scores from predicted label maps. Evaluation setups must also state how classes absent from an image are handled, because averaging an undefined IoU can otherwise distort the result. mIoU therefore rewards models that delineate every meaningful category accurately, not just the easiest pixels.

Mean Intersection over Union (mIoU) is a segmentation metric that averages the Intersection over Union (IoU) across classes. For each class, IoU is the overlap between predicted and ground-truth pixels divided by their combined area. mIoU measures both false positives and false negatives, providing a balanced assessment of mask quality. It is a standard benchmark for comparing semantic segmentation models, especially when class frequencies differ.

Imagine comparing two cut-out shapes: one drawn by a person, and one made by an AI. Mean Intersection over Union (mIoU) tells you how well those shapes overlap.

In image segmentation, the AI labels each pixel—for example, marking roads, cars, trees, and people. For each kind of object, mIoU checks how much of the AI’s marked area matches the correct area, while penalizing areas it missed or labeled wrongly. It then averages the result across all object types. A score closer to 1, or 100%, means the AI’s pixel-by-pixel understanding is more accurate.