Anchor Box

When a detector scans a street photo, it needs a practical starting guess for where cars, people, and signs could be. An anchor box is that starting guess: a set of pre-placed reference rectangles spread across an image or its feature map.

Each anchor has a position, width, height, and aspect ratio. Rather than inventing every possible bounding box from scratch, an anchor-based model asks two focused questions for each reference rectangle: “Does this contain an object?” and “How should I shift and resize it to fit the object?” The model predicts small numeric adjustments—such as moving the center left or making the box taller—then produces a refined bounding box and a class label. It is like beginning with a sheet of differently shaped picture frames, then adjusting the best frame around each visible object.

During training, anchors are compared with the human-labeled boxes using intersection over union (IoU). Anchors with high overlap become positive examples; anchors with little overlap become background examples. A detector commonly uses multiple scales and shapes at every location so it can cover varied objects:

• Wide, short anchors help fit vehicles or text lines.
• Tall, narrow anchors help fit standing people.
• Larger anchors help detect nearby objects; smaller ones help find distant objects.

Anchors made fast detectors such as Faster R-CNN, SSD, and early YOLO versions effective because they turn open-ended localization into predictable regression from useful starting points. Poorly chosen anchor sizes can miss tiny defects in production-line inspection, crowd tightly packed faces, or struggle with unusually long medical structures. Detectors also generate many overlapping refined boxes, so non-maximum suppression removes duplicates. Newer anchor-free detectors avoid these predefined rectangles, but anchors remain an important way to understand how many widely used object detectors locate objects.