Teaching Machines to See Defects

A demo of visual inspection looks easy: collect a few hundred images, fine-tune a detector, draw green boxes. A production line is where that story falls apart. The camera shakes, the lighting drifts with the seasons, and the defect you care about appears eleven times a month.

After working on inspection systems in real plants, I have become convinced that model architecture is maybe the fourth most important decision.

Lighting is the dataset

Two images of the same part under two light angles are more different to a convnet than two different parts under identical light. If the illumination is not controlled, the model spends its capacity learning the fixture instead of the flaw.

• Fix the geometry first: dome, coaxial, or low-angle dark-field, chosen for the surface, not for the camera budget.
• Log the exposure and gain with every frame. A drift alert is cheaper than a silent accuracy collapse.
• Deliberately collect the ugly cases — glare, oil, dust, a shifted part — and label them as their own conditions.

Taxonomy before training

"Defect" is not a class. A scratch, a dent, a contaminant, and a print misregistration have different causes, different owners, and different costs. Agreeing on the label tree with the quality team is unglamorous work that determines whether the output is actionable. If the model says "bad part" and nobody knows which station to adjust, the system produces alerts, not value.

The asymmetric cost

False positives stop the line. False negatives ship to a customer. These are not equally bad, so a single accuracy number is actively misleading. Define the operating point with the people who pay for each error, then hold it fixed while you iterate on everything else.

CopyEscapes per million      -> business cost, not a metric
Reviewer minutes / hour  -> the real capacity limit
Recall on critical class -> the number to protect in CI

In practice this means thresholding per class, routing ambiguous cases to a human, and treating the review queue as a first-class product surface. The review UI is where most of the daily value is created.

When normal data is all you have

Critical defects are rare by definition, so supervised detection often starves. Anomaly-detection approaches — a compact embedding plus a density model of good parts — let you ship with only healthy examples, then expand the labelled set as real defects are found. It is not magic: it is sensitive to domain shift and needs a calibration set that is refreshed whenever the process changes.

The camera is a sensor, the model is a detector, but the inspection system is the whole loop: light, label, decide, review, improve.

From a whiteboard that survived three factory visits

What I would build first

If starting again, I would spend the first two weeks on a disciplined data pipeline: versioned captures, condition metadata, a review tool, and a small frozen evaluation set that never changes. The model can be swapped in an afternoon. A trustworthy loop cannot.

Computer vision in industry is less about perception and more about building an organisation that can agree on what it saw.