DINO (Self-Supervised)

DINO is a way to teach an image model useful visual concepts without giving it human-written labels such as “cat,” “car,” or “tumor.” Instead, it learns that several altered views of the same image should lead to a similar understanding—much like recognizing a dog whether it is cropped, blurred slightly, or viewed in different lighting.

DINO stands for self-distillation with no labels. It uses two copies of a vision model: a student and a teacher. The system creates multiple augmented versions of one image, including large “global” crops and smaller “local” crops. The student tries to match the teacher’s output distribution for these views. Crucially, the teacher is not trained directly with gradients; its weights are a slowly updated moving average of the student’s weights. This creates a stable target for learning.

• Augmentations force the model to ignore unimportant changes such as color shifts or cropping.
• Centering and sharpening prevent a useless solution where every image receives the same representation.
• No labeled examples or manually selected “negative” image pairs are required.

When trained with a Vision Transformer (ViT), DINO develops feature representations that capture object shape, parts, and scene structure. Its attention maps can highlight the main object in an image even though the model was never given segmentation masks. A DINO-pretrained backbone can then be adapted for image classification, object detection in retail photos, medical-image segmentation, visual inspection of manufacturing defects, or video understanding. Without this kind of pretraining, these tasks need far more labeled data and the resulting model is less robust to visual variation. DINO showed that learning from images themselves can produce features that transfer remarkably well to many downstream vision problems.