Qwen2.5 14B Instruct
Q4_K - Medium
14.8Bparams
COMPARE ACCELERATORS
87 accelerators tested
Select Accelerators
NVIDIA GeForce RTX 5090
31GB
NVIDIA H100 PCIe
79GB
NVIDIA GeForce RTX 4090
24GB
NVIDIA GeForce RTX 4080
16GB
NVIDIA GeForce RTX 3090 Ti
24GB
Qwen2.5 14B Instruct - Q4_K - Medium
LEADERBOARD
GPU / 16GB
PROMPT
2454
tokens/s
GENERATION
50.4
tokens/s
TTFT
535
ms
LOCALSCORE
614
GPU / 48GB
PROMPT
2936
tokens/s
GENERATION
28.1
tokens/s
TTFT
471
ms
LOCALSCORE
560
GPU / 16GB
PROMPT
2424
tokens/s
GENERATION
29.9
tokens/s
TTFT
555
ms
LOCALSCORE
507
GPU / 12GB
PROMPT
1903
tokens/s
GENERATION
31.9
tokens/s
TTFT
691
ms
LOCALSCORE
443
GPU / 20GB
PROMPT
1408
tokens/s
GENERATION
31.4
tokens/s
TTFT
952
ms
LOCALSCORE
359
GPU / 11GB
PROMPT
1166
tokens/s
GENERATION
42.2
tokens/s
TTFT
1.08
sec
LOCALSCORE
357
GPU / 16GB
PROMPT
1248
tokens/s
GENERATION
26.7
tokens/s
TTFT
1.10
sec
LOCALSCORE
312
GPU / 20GB
PROMPT
1037
tokens/s
GENERATION
24.9
tokens/s
TTFT
1.34
sec
LOCALSCORE
268
GPU / 512GB
PROMPT
579
tokens/s
GENERATION
35.9
tokens/s
TTFT
2.03
sec
LOCALSCORE
217
GPU / 16GB
PROMPT
733
tokens/s
GENERATION
21.0
tokens/s
TTFT
1.83
sec
LOCALSCORE
203
GPU / 96GB
PROMPT
445
tokens/s
GENERATION
34.4
tokens/s
TTFT
2.67
sec
LOCALSCORE
179
GPU / 128GB
PROMPT
290
tokens/s
GENERATION
27.8
tokens/s
TTFT
4.04
sec
LOCALSCORE
126