Edge Devices

On-Device Inference

6.3Score

Privacy: Local Only
Power: 15W
Budget: $ · Budget
Composite Score: 6.3 / 10

Composition

NVIDIA Jetson Orin32GB · 64GB eMMC
—Score

Example System

NVIDIA Jetson Orin
32GB
64GB eMMC

90-day Benchmarks

Llama 3.1 70B Q4_K_M tokens/sec62.1 tokens/sec

Qwen2.5 32B Q4_K_M tokens/sec62.8 tokens/sec