AI on The Edge

Benchmarks

2 workloads

Llama 3.1 70B Q4_K_M tokens/sec

SetupLatest score (tokens/sec)Last runSamples (90d)
GPU Rigs92.4May 5, 202613
Apple Silicon87.7May 5, 202613
Mini PCs80.8May 5, 202613
Edge Devices64.5May 5, 202613

Qwen2.5 32B Q4_K_M tokens/sec

SetupLatest score (tokens/sec)Last runSamples (90d)
GPU Rigs94.6May 5, 202613
Apple Silicon89.4May 5, 202613
Mini PCs78.8May 5, 202613
Edge Devices64.3May 5, 202613