Edge Accelerators on the Factory Floor: Vision Inference Without a Server Room

Overview

Visual inspection is the most widely deployed industrial AI workload in India, and the one where centralised GPU thinking fails most clearly. A line running at speed needs a verdict on every part, within a fixed cycle time, at every station. Streaming footage to a server room — or to a cloud region — adds latency you cannot control and bandwidth you should not be paying for. Purpose-built edge accelerators in a single-digit-watt envelope let inference sit in the cabinet beside the camera. This article covers when that architecture is right, how to size it, and the honest trade-offs.

Cycle time, not throughput, is the requirement

An inspection system is judged by whether it returns a decision before the part moves on. If the station beat is 200 ms, a model averaging 80 ms with occasional 400 ms excursions is a failure — the excursions become escaped defects or false rejects. This is why edge inference is a determinism problem rather than a throughput problem, and why datasheet TOPS is a weak predictor. Specify worst-case latency at your resolution and model, then verify it on the actual line, not on a benchmark image set.

Why the GPU instinct misleads here

GPUs earn their efficiency by batching many samples together. At an inspection station there is one camera producing one frame that must be judged now — batch size one. The GPU spends much of its time moving weights rather than computing, so the achieved fraction of its rated performance is poor, while it still draws hundreds of watts and needs active cooling and conditioned power in an environment that has neither. Multiply that across twenty stations and the architecture is wrong on latency, on watts and on cost. Our edge accelerator versus GPU comparison works through the underlying reasons.

Sizing: per station or per line

As a rule, start per-station where the beat is tight and consolidate only when measurement shows headroom. Consolidation looks efficient on a spreadsheet and is the usual source of intermittent latency excursions in production.

The environment decides more than you expect

Factory cabinets are hot, dusty and often sealed, and the power available at a station is not data-centre power. A device rated at roughly 2.5W to 9W can run passively cooled in that enclosure; a card needing hundreds of watts and active airflow cannot, without building infrastructure around it. Fanless operation also removes a moving part from an environment where filters clog. When comparing candidates, weight the thermal and ingress story alongside the performance figure — the deployment that survives three years in a plant is rarely the one with the highest peak TOPS.

Keeping factory data on site

Inspection footage frequently captures more than parts: operators, badges, and in some plants customer or patient-linked material. Processing at the camera means the video stream never leaves the site, and only verdicts and metrics travel upstream. Under the DPDP Act — penalties up to ₹250 crore per violation — that is the cleanest position available, because it removes the transfer question rather than answering it. It also removes a large, continuous bandwidth cost, which matters for plants on constrained links. The retraining loop can still run centrally in-country, as covered in our on-prem sizing guide.

What edge accelerators will not do

Be clear about the limits. These are inference-only devices: they do not train or fine-tune, so every model change still requires GPU capacity somewhere. They favour quantised INT8 or INT4 vision networks, so a model that only performs acceptably at FP16 may not port cleanly. And toolchain maturity varies more between vendors than raw specifications do — compilation quality determines how much of the rated performance you actually see. Budget engineering time for quantisation and validation, and prove the pipeline on one line before rolling out to twenty.

Frequently asked questions