Image Annotation for Computer Vision

What is image annotation

Image annotation is the work of marking what each photo contains and giving it a name: this is a tray, this is a hand, this is a defect. From those examples, a computer vision model learns to recognize the same things in images it has never seen. It is also called image labeling or data labeling for computer vision, and it is usually the least visible and most labor-intensive part of a vision project.

How an image is annotated

The type of label depends on what the model has to do:

• Classification: one label for the whole image, for example correct or defective.
• Detection: a box around each object, with its class.
• Segmentation: the exact outline of the object, pixel by pixel.
• Keypoints: specific positions, such as a person's joints to estimate their pose.

Before starting, you need to fix the classes and a shared standard. If two people label the same photo differently, the model learns that doubt. That is why teams write a guide with examples of what counts, what does not and the gray areas, and review a share of the labels.

A simple example

An overhead camera watches a packaging line. For the system to detect badly fitted caps, you collect photos of correct and defective packages from several shifts and batches, draw a box over each cap and mark whether it is right or wrong. Part of those photos is set aside and not used for training: it serves to test the model on images it does not know.

How many images to train a model

There is no universal number. It depends on how much the product or scene varies, the size of what has to be detected, the type of model and the lighting and camera conditions. Covering the real variations (shifts, batches, reflections, angles) matters more than piling up many identical photos. And it pays to separate training, validation and test sets, so the result does not look better than it will be in production.

How we apply it at DPI

At DPI Systems, annotation is part of our computer vision projects. In MBO Smart Buffet, the computer vision system we built to measure consumption at hotel buffets, we work with 5,033 labeled images from real buffets, from 23 cameras. The detector only learns classes that do not change with the menu, such as tray, tongs, hand or guest. To know which dish is on each tray, Meta's DINOv3 model computes a fingerprint of 384 numbers that is searched in Qdrant. So adding a new dish means photographing it, not retraining or relabeling. If the system does not recognize a dish, it records it as unknown to be labeled later.