NVIDIA Jetson Deep Dive: Orin Family and Jetson Thor for Edge AI
Overview
NVIDIA Jetson is the default general-purpose platform for edge AI, and the reason is software as much as silicon. The Orin family spans vendor-stated 67 INT8 sparse TOPS at 7-25 W (Orin Nano Super) to 275 INT8 sparse TOPS at 15-60 W (AGX Orin), and the Blackwell-generation Jetson Thor modules extend the line into FP4 transformer territory at up to a vendor-stated 2070 FP4 TFLOPS. Every module runs the same CUDA, TensorRT and JetPack stack used by an ecosystem NVIDIA reports at around 2 million developers. This article covers the module line-up, power behaviour, software advantage and the honest cases where Jetson is the wrong answer.
Key takeaways
- Orin spans three tiers: Orin Nano Super (vendor-stated 67 INT8 sparse TOPS, 7-25 W), Orin NX (up to 157 INT8 sparse TOPS, 10-40 W) and AGX Orin (up to 275 INT8 sparse TOPS, 15-60 W).
- Sparse versus dense matters: AGX Orin’s headline 275 TOPS assumes 2:4 structured sparsity; NVIDIA’s dense INT8 figure is 138 TOPS.
- Jetson Thor moves to Blackwell: the T5000 module is vendor-stated at up to 2070 FP4 TFLOPS with 128 GB of memory, configurable between roughly 40 and 130 W.
- The moat is JetPack, CUDA and TensorRT plus around 2 million reported developers; almost any model that runs on a datacentre NVIDIA GPU can be ported.
- For a single fixed CNN workload under 5 W, a dataflow accelerator usually beats Jetson on efficiency and unit cost.
The Orin family: three tiers, one architecture
All Orin modules pair Arm Cortex-A78AE CPU cores with an Ampere-generation GPU carrying tensor cores, differing in core counts, memory and power ceilings. The Orin Nano Super (8 GB LPDDR5, 1,024 CUDA cores) is vendor-stated at 67 INT8 sparse TOPS after the late-2024 JetPack power-mode uplift, with a developer kit priced at 249 US dollars. Orin NX modules reach a vendor-stated 157 INT8 sparse TOPS at 10-40 W with 16 GB. AGX Orin tops out at 275 INT8 sparse TOPS with up to 64 GB and 15-60 W configurable power. The sparse qualifier is important: headline figures assume 2:4 structured sparsity, and NVIDIA’s own material puts dense INT8 peak at 138 TOPS on AGX Orin. Treat all of these as vendor-stated peaks, not delivered throughput.
Jetson Thor: the Blackwell generation
Announced in 2025 for physical AI and robotics, Jetson Thor pairs a Blackwell-architecture GPU with a 14-core Arm Neoverse-V3AE CPU. The Jetson T5000 module is vendor-stated at up to 2070 FP4 TFLOPS with 128 GB of memory and power configurable between roughly 40 and 130 W; the T4000 is vendor-stated at up to 1200 FP4 TFLOPS with 64 GB. Note the precision shift: Thor’s headline numbers are FP4, not INT8, so they cannot be compared arithmetically against Orin’s INT8 figures. Thor targets multi-sensor robotics and on-device reasoning models rather than replacing Orin in cost-sensitive vision products, and NVIDIA sells it alongside Orin rather than instead of it. The AGX Thor developer kit launched at 3,499 US dollars.
Power modes: TOPS are configurable, not constant
Every Jetson exposes named power modes (for example 7 W, 15 W and 25 W on Orin Nano Super; up to 60 W on AGX Orin; up to about 130 W on Thor) that cap CPU and GPU clocks. Peak TOPS are only available at the top mode with adequate cooling; a passively cooled enclosure in a 45 degree Celsius Indian roadside cabinet will realistically run one or two modes down. Budget thermals for the mode you intend to sustain, not the datasheet peak.
The software advantage: JetPack, CUDA, TensorRT
JetPack bundles Ubuntu-based Linux, CUDA, cuDNN, TensorRT, DeepStream for video pipelines and Isaac for robotics. Because the same CUDA and TensorRT APIs run in the datacentre, models developed on any NVIDIA GPU port to Jetson with quantisation and optimisation rather than rewrites, and new operators typically appear on Jetson long before specialist compilers support them. That flexibility carries overhead, which is why per-watt comparisons favour fixed-function parts on static workloads, as discussed in edge accelerators versus GPUs. NVIDIA reports around 2 million developers in the Jetson ecosystem, which in practice means abundant hiring, community debugging and third-party libraries.
Form factor: SOM plus carrier board
Jetson ships as a system-on-module with a standardised connector; you buy or design a carrier board for your I/O. A large partner network (Connect Tech, Aetina, Advantech, Seeed and others) sells carriers and ruggedised systems, which shortens hardware development to carrier selection for most products. Pin-compatibility within a family lets one carrier serve Orin Nano and Orin NX, giving a performance upgrade path without a board respin.
Jetson module line-up at a glance
| Module | Vendor-stated peak AI compute | Memory | Power range | Typical role |
|---|---|---|---|---|
| Orin Nano Super | 67 TOPS (INT8, sparse) | 8 GB LPDDR5 | 7-25 W | Entry vision, single/few streams, developer kits |
| Orin NX 16GB | 157 TOPS (INT8, sparse) | 16 GB LPDDR5 | 10-40 W | Multi-stream vision, compact industrial boxes |
| AGX Orin 64GB | 275 TOPS (INT8, sparse); 138 dense | 64 GB LPDDR5 | 15-60 W | Robotics, multi-model pipelines, small LLMs |
| Jetson T4000 (Thor) | 1200 TFLOPS (FP4) | 64 GB | Configurable, Thor-class | Robotics and physical AI, mid tier |
| Jetson T5000 (Thor) | 2070 TFLOPS (FP4) | 128 GB | About 40-130 W | Humanoids, multi-sensor reasoning, on-device VLMs |
When Jetson is the right answer, and when it is not
Choose Jetson when workloads are heterogeneous or evolving, when you need transformers or VLMs at the edge, when time-to-market depends on reusing datacentre models, or when the team already knows CUDA. Choose a fixed-function accelerator instead when the workload is a stable CNN pipeline, the power budget is under about 5 W, or unit cost at fleet scale dominates: parts covered in the Hailo deep dive and the specialist silicon survey often deliver better inference per watt and per dollar there. For a structured decision path, see the vendor-agnostic selection framework.
Frequently asked questions
What is the difference between sparse and dense TOPS on Jetson?
Sparse figures assume 2:4 structured sparsity, where half the weights are zero and skipped by the tensor cores. Most off-the-shelf models are dense, so the dense figure (138 INT8 TOPS on AGX Orin, per NVIDIA) is the more realistic ceiling unless you specifically prune for sparsity.
Does Jetson Thor replace AGX Orin?
No. NVIDIA positions Thor for physical AI, robotics and on-device reasoning models, and continues to sell the Orin family for mainstream embedded vision. Thor’s FP4-headline compute and 40-130 W envelope address a different class of workload than a 15-60 W Orin.
Can Jetson run large language models locally?
Within memory limits, yes. AGX Orin’s 64 GB and Thor’s 128 GB allow quantised small and mid-size LLMs and VLMs via TensorRT-LLM and community runtimes. Throughput depends heavily on quantisation; see our guide to INT8 and INT4 quantisation for the trade-offs.
Why does my model deliver far fewer TOPS than the datasheet?
Datasheet TOPS are theoretical peaks at maximum clocks and ideal utilisation. Memory bandwidth, operator mix, preprocessing and thermal throttling all reduce delivered throughput. Benchmark your own model at your intended power mode before sizing a fleet.
Is Jetson viable for large fleet deployments in India?
Yes, with planning: modules carry long lifecycle commitments and wide distributor availability, but thermal derating in hot enclosures and per-unit cost versus fixed-function parts should be modelled first. Our article on deploying edge AI fleets in India covers provisioning, OTA and DPDP considerations.
Ready to deploy?
Talk to an RDP architect about power, cooling and lead time.