Hailo-8 and Hailo-10H Deep Dive: Dataflow Silicon for Edge Inference

Overview

The Hailo-8 and Hailo-10H are dedicated edge inference processors from Hailo, a fabless silicon company focused entirely on neural network acceleration. The Hailo-8 delivers a vendor-stated 26 TOPS (INT8) at a typical 2.5 W by executing networks from on-chip SRAM with no external DRAM. The newer Hailo-10H adds a direct DDR interface to 4 GB or 8 GB of LPDDR4X and a published 40 TOPS (INT4) – the change that makes small language models and vision-language models practical on-device. This deep dive covers the dataflow architecture, what the published figures do and do not mean, form factors, the Dataflow Compiler and HailoRT toolchain, and where these parts genuinely fit.

The Hailo family at a glance

Hailo ships three relevant lines. The Hailo-8 (vendor-stated 26 TOPS INT8) and its cut-down sibling the Hailo-8L (13 TOPS INT8) are pure computer-vision accelerators. The Hailo-10H is the generative-AI part: a second-generation neural core plus a DDR controller, published at 40 TOPS (INT4) with industrial (-40 to +85 C) and automotive temperature grades. Separately, the Hailo-15 family integrates a similar neural core into a camera SoC with Arm CPU cores – a different product class aimed at smart cameras rather than host-attached acceleration. All of them are inference-only: there is no training mode and no general-purpose FP32 path.

A dataflow architecture, not a shrunken GPU

A GPU executes kernels against a large external memory (GDDR or HBM), and most of its energy budget goes into moving tensors across that boundary. Hailo’s structure-defined dataflow architecture inverts this: the Dataflow Compiler analyses the neural network graph at build time and maps its layers onto a pipeline of compute, memory and control elements distributed across the die. Weights and activations for the compiled network live in on-chip SRAM, so at runtime data streams through a fixed layer pipeline instead of bouncing to DRAM between kernels. Two practical consequences follow. First, power: the vendor-stated 2.5 W typical figure for both Hailo-8 and Hailo-10H is credible precisely because DRAM traffic is eliminated or minimised. Second, determinism: because the schedule is fixed at compile time, per-frame latency is highly repeatable – valuable for industrial pipelines with hard cycle budgets, as discussed in our manufacturing vision AI deployment playbook. The trade-off is rigidity: the chip runs what the compiler can map, and nothing else.

Why the Hailo-10H’s DRAM interface changes the workload class

The Hailo-8’s SRAM-only design caps the size of model that fits – fine for detection, segmentation and classification CNNs, a hard wall for language models whose weights run into gigabytes even after quantisation. The Hailo-10H’s direct DDR interface to 4 GB or 8 GB of LPDDR4X removes that wall: weights stream from local DRAM owned by the accelerator, not borrowed from the host. Hailo’s published demonstrations include ~2B-parameter LLMs and vision-language models running on-device within the same low single-digit-watt envelope. Expectations should stay calibrated: reported first-token latency and tokens-per-second are modest compared with any discrete GPU, and context lengths are constrained by that same DRAM. The realistic workload class is compact SLMs, VLM-based scene description, and multimodal pipelines where video analytics and a small language head run together – not chatbot-grade serving.

Form factors and integration

Both chips are host-attached devices, and module options are broad:

• M.2 2280 modules in B+M, M and A+E key variants, using PCIe Gen 3 x2 (x4 on the M-key Hailo-8 module) – the most common route into gateways and industrial PCs.
• Half-mini PCIe and full PCIe card options through Hailo and board partners such as AAEON and UP.
• A USB 3.1 Gen 2 Hailo-10H variant from partners, plus Raspberry Pi 5 AI HAT boards – useful for prototyping fleets cheaply.
• Industrial temperature grades and fanless-friendly thermals; a small heatsink is typically all a 2.5 W part needs inside a cabinet.

The host still matters: it runs the OS, decodes video, executes any layers the compiler could not map, and feeds the accelerator over PCIe.

Toolchain: Dataflow Compiler, HailoRT and the Model Zoo

Deployment is a compile-first workflow. The Hailo AI Software Suite ingests ONNX or TensorFlow models, quantises them to INT8/INT4 against a calibration dataset you supply, and emits a compiled HEF binary. HailoRT then loads and schedules that binary from C, C++ or Python, with GStreamer-based reference pipelines (TAPPAS) for multi-stream video. A Model Zoo of pre-compiled detection, segmentation and pose networks shortens time-to-first-inference considerably. Budget engineering time honestly: quantisation accuracy must be validated per model, unsupported operators fall back to the host CPU, and compiled throughput can differ meaningfully from datasheet TOPS depending on how well a network maps to the pipeline.

Hailo-8 vs Hailo-10H

Where Hailo fits – and honest limits

Hailo’s sweet spot is fixed-function inference at scale: multi-camera analytics in retail and traffic systems, quality inspection on production lines, and on-device VLM summarisation where sending video off-site is unattractive – in India, keeping footage on-premises also simplifies obligations under the DPDP Act. The limits are equally clear: no training, mandatory quantisation, a smaller ecosystem than CUDA, and per-model compiler work that a GPU deployment avoids. For when that trade goes the other way, see edge AI accelerators vs GPUs, and for how Hailo stacks against its direct rivals, our Hailo vs Axelera vs Blaize comparison.

Frequently asked questions