Synthetic Data for Industrial Vision: GPU Budgets and Sim-to-Real

Overview

Industrial defect detection has a data problem that more cameras cannot solve: the defects you most need to catch are the ones that happen rarely, so you have almost no examples of them. A line running at 99.7 percent yield produces three defective parts per thousand, spread across a dozen defect types, which is nowhere near enough to train a reliable detector on the rare classes. Synthetic data generation — rendering defects into virtual scenes — addresses this directly, and in 2026 it is a standard part of industrial vision programmes. This article covers where the GPU budget goes and what can go wrong.

Why the data does not exist

Consider a plant wanting to detect twelve defect classes. Some, like surface scratches, occur often enough to gather hundreds of examples in a month. Others, like a specific weld porosity pattern, might occur twice a year. Conventional collection would require years, and by then the process will have changed. Worse, the rare defects are frequently the expensive ones — the ones that cause a recall or a customer rejection — so the classes with the least data carry the most business risk.

Augmentation of existing images helps marginally but cannot invent a defect type that is absent. What synthetic generation offers is the ability to create the class deliberately: model the part, model the defect physically, render it under many conditions, and produce thousands of labelled examples of something that has occurred twice in production.

Where the compute actually goes

Teams consistently misplace the budget here. Training a defect detection model on a mixed dataset of tens of thousands of images is a modest job — hours on a single workstation-class GPU. Generating those images is the expensive part: physically based rendering with realistic materials, lighting and camera models, repeated across many randomised configurations, is a graphics workload measured in GPU-days for a substantial dataset.

The practical implication is that the hardware profile for a synthetic data programme looks more like a rendering farm than a training cluster. Ray tracing throughput, memory for scene assets, and the ability to run many render jobs in parallel matter more than tensor core FLOPS. It is also bursty: heavy generation when a new part or defect class is introduced, then quiet. That profile suits shared or rented capacity for the generation phase with owned hardware for training and inference.

Managing the domain gap

The pattern across every row is that deliberate variation, not photorealism, is the mechanism. A perfectly photorealistic render of one lighting condition teaches the model that condition; a moderately realistic render across a wide distribution teaches it invariance. This is the same domain randomisation principle used in robotics, covered in physical AI in Indian factories.

The mixing ratio

Synthetic-only training reliably disappoints on a real line. The productive approach mixes: use all available real images, supplement rare classes with synthetic examples, and weight the training so the model does not overfit the synthetic distribution. A common starting point is to bring every class up to a comparable count using synthetic data, then fine-tune the final stage on real images only, which pulls the model back toward the real distribution.

Two practical cautions. Keep a strict separation between synthetic and real in the dataset manifest so you can always ablate. And retain the generation parameters alongside the images, because when the model fails on a particular condition you will want to know whether that condition was in the generated distribution at all.

Validation that means something

The single most common methodological error is evaluating on synthetic test data. A model trained on synthetic images will score well on synthetic images regardless of whether it works on the line, and that result is worse than no result because it creates false confidence.

The valid protocol is a held-out set of real images, including real examples of the rare classes even if there are only a handful, plus a shadow-mode deployment where the model runs on the live line without acting and its outputs are compared against inspector decisions for a period. Only after shadow mode agrees with human inspection at an acceptable rate should the system influence disposition. The inspection pipeline design is covered in the manufacturing vision AI GPU server playbook.

Practical sequencing for an Indian plant

Start where real data exists. Build and deploy a detector for the common defect classes first, using real images, and get it into shadow mode. This proves the pipeline, the camera positioning and the integration with line control before adding synthetic complexity. Then add synthetic generation for the rare classes that the deployed system is demonstrably missing.

Two India-specific notes. CAD models of parts are often held by the customer rather than the contract manufacturer, so securing model access is a commercial step that should happen early. And rendering capacity is a good candidate for subsidised national compute or rented capacity given its bursty profile — IndiaAI empanelled GPUs at rates near a dollar per GPU-hour suit a generation campaign far better than buying a render farm that idles between part introductions. The wider planning frame is in the manufacturing vision AI edge deployment playbook.

Frequently asked questions