| Hardware profile | ||||
|---|---|---|---|---|
| Hardware | Memory | Bandwidth | Computing power | vLLM |
| RTX PRO 6000NVIDIA RTX PRO 6000 Blackwell series | 96 GBGPU VRAM | 1792 GB/s | ~3511 TOPS | yes |
| Compatible models Estimated speeds, not benchmark results: calculated from memory bandwidth and model size. Real results can differ significantly because there is no precise formula for deriving LLM generation speed from hardware specifications alone. | |||
|---|---|---|---|
| Model | Size | Approx. Q4 memory | Estimated generation |
| Mistral Medium 3.5Dense | 128B | 76.8 GB | 12 tok/s4-bit quantization |
| Qwen 3.8 Flash NextMoE | 125Bactive 6B | 75 GB | 159 tok/s4-bit quantization |
| Ling 3.0 Flash VLMoE | 124Bactive 5.5B | 74.4 GB | 169 tok/s4-bit quantization |
| Ling 3.0 FlashMoE | 124Bactive 5.1B | 74.4 GB | 179 tok/s4-bit quantization |
| Qwen 3.5 122BMoE | 122Bactive 10B | 73.2 GB | 105 tok/s4-bit quantization |
| Ornith 1.5 122BMoE | 122Bactive 9.9B | 73.2 GB | 106 tok/s4-bit quantization |
| Nemotron 3 SuperMoE | 120Bactive 12B | 72 GB | 90 tok/s4-bit quantization |
| Nemotron 3.5 Super VLMoE | 120Bactive 12B | 72 GB | 90 tok/s4-bit quantization |
| Mistral Small 4MoE | 119Bactive 6.5B | 71.4 GB | 149 tok/s4-bit quantization |
| Laguna S 2.1MoE | 118Bactive 8B | 70.8 GB | 126 tok/s4-bit quantization |
| GPT-OSS 120BMoE | 116.8Bactive 5.1B | 70.1 GB | 179 tok/s4-bit quantization |
| Sarvam 105BMoE | 105Bactive 10.3B | 63 GB | 102 tok/s4-bit quantization |
| Qwen 3 Coder Next 80BMoE | 80Bactive 3B | 48 GB | 256 tok/s4-bit quantization |
| Kolibri 1MoE | 78Bactive 3.5B | 46.8 GB | 234 tok/s4-bit quantization |
| Nemotron 3.5 BoltMoE | 74Bactive 9B | 44.4 GB | 115 tok/s4-bit quantization |
| Ornith 1.0 35BMoE | 35Bactive 3B | 21 GB | 256 tok/s4-bit quantization |
| Ornith 1.5 35BMoE | 35Bactive 3B | 21 GB | 256 tok/s4-bit quantization |
| Qwen 3.6 35BMoE | 35Bactive 3B | 21 GB | 256 tok/s4-bit quantization |
| Laguna XS 2.1MoE | 33Bactive 3B | 19.8 GB | 256 tok/s4-bit quantization |
| Qwen 2.5 32BDense | 32.5B | 19.5 GB | 48 tok/s4-bit quantization |
| Gemma 4 31BDense | 31B | 18.6 GB | 50 tok/s4-bit quantization |
| Qwen 3 Coder 30BMoE | 30.5Bactive 3.3B | 18.3 GB | 242 tok/s4-bit quantization |
| Granite 4.1 30BDense | 30B | 18 GB | 52 tok/s4-bit quantization |
| Granite 4.2 30BDense | 30B | 18 GB | 52 tok/s4-bit quantization |
| Muse GlimmerDense | 30B | 18 GB | 52 tok/s4-bit quantization |
| GLM 4.7 FlashMoE | 30Bactive 3B | 18 GB | 256 tok/s4-bit quantization |
| Nemotron 3.5 LightningMoE | 30Bactive 3B | 18 GB | 256 tok/s4-bit quantization |
| Fara 1.5 27BDense | 27B | 16.2 GB | 58 tok/s4-bit quantization |
| Qwen 3.6 27BDense | 27B | 16.2 GB | 58 tok/s4-bit quantization |
| Qwen 3.8 27BDense | 27B | 16.2 GB | 58 tok/s4-bit quantization |
| Gemma 4 26BMoE | 26Bactive 4B | 15.6 GB | 213 tok/s4-bit quantization |
| LFM 2.5 24BMoE | 24Bactive 2B | 14.4 GB | 323 tok/s4-bit quantization |
| GPT-OSS 20BMoE | 20.9Bactive 3.6B | 12.5 GB | 228 tok/s4-bit quantization |
| Instella-MoE 16B ThinkMoE | 16Bactive 2.8B | 9.6 GB | 267 tok/s4-bit quantization |
| Qwen 3 14BDense | 14B | 8.4 GB | 107 tok/s4-bit quantization |
| Gemma 3 12BDense | 12B | 7.2 GB | 124 tok/s4-bit quantization |
| Gemma 4 12BDense | 12B | 7.2 GB | 124 tok/s4-bit quantization |
| Fara 1.5 9BDense | 9B | 5.4 GB | 160 tok/s4-bit quantization |
| Ornith 1.0 9BDense | 9B | 5.4 GB | 160 tok/s4-bit quantization |
| Ornith 1.5 9BDense | 9B | 5.4 GB | 160 tok/s4-bit quantization |
| Qwen 3.5 9BDense | 9B | 5.4 GB | 160 tok/s4-bit quantization |
| Qwythos 9BDense | 9B | 5.4 GB | 160 tok/s4-bit quantization |
| Granite 4.1 8BDense | 8B | 4.8 GB | 177 tok/s4-bit quantization |
| Granite 4.2 8BDense | 8B | 4.8 GB | 177 tok/s4-bit quantization |
| Qwen 3 8BDense | 8B | 4.8 GB | 177 tok/s4-bit quantization |
| LFM 2.5 8BMoE | 8Bactive 1B | 4.8 GB | 435 tok/s4-bit quantization |
| Ling 3.0 TinyMoE | 7.9Bactive 1.3B | 4.7 GB | 394 tok/s4-bit quantization |
| Fara 1.5 4BDense | 4B | 2.4 GB | 313 tok/s4-bit quantization |
| Gemma 3 4BDense | 4B | 2.4 GB | 313 tok/s4-bit quantization |
| Qwen 3 4B Instruct 2507Dense | 4B | 2.4 GB | 313 tok/s4-bit quantization |
| Qwen 3.5 4BDense | 4B | 2.4 GB | 313 tok/s4-bit quantization |
| Spark X2.5 4BDense | 4B | 2.4 GB | 313 tok/s4-bit quantization |
| Granite 4.1 3BDense | 3B | 1.8 GB | 388 tok/s4-bit quantization |
| Granite 4.2 3BDense | 3B | 1.8 GB | 388 tok/s4-bit quantization |
| Llama 3.2 3B InstructDense | 3B | 1.8 GB | 388 tok/s4-bit quantization |
| Ministral 3 3BDense | 3B | 1.8 GB | 388 tok/s4-bit quantization |
| Nanbeige 4.2 3BDense | 3B | 1.8 GB | 388 tok/s4-bit quantization |
| LFM 3 1.7BDense | 1.7B | 1 GB | 560 tok/s4-bit quantization |
Compares updated Strata Coder and Turbo Brilliance via 9 tests: speed, strategy game, tool calling, CAPTCHA, 262K context, SVG, voxels, motion design and physics puzzles.
Tests Strata Coder IQ1_M against Qwen 3.8 Flash Next NVFP4 using 9 benchmarks: speed, battle strategy, tool use, visual CAPTCHAs, long-context retrieval, SVG, voxels, motion and physics.
Tests ThinkingCap and Swift 1.5 Qwen 3.8 27B fine-tunes in GGUF Q4_K_M on an RTX PRO 6000 using llama.cpp across nine benchmarks covering speed, tool calling, and visual tasks.
Compares ukisai's Swift 1.5 NVFP4 and RadixArk's Qwen 3.8 27B NVFP4 via 10 tests on an RTX PRO 6000 using SGLang, covering speed, tool calling, 262K context, games, SVG, video editing, voxels, motion
Tests Qwen3.8 variants on RTX PRO 6000 via SGLang/vLLM across 10 tasks including BFCL, long context, SVG, video editing, voxels, motion design, Rube Goldberg physics and CAPTCHAs.
Tests Qwen3.8-Flash-Next on RTX PRO 6000 with SGLang, measuring speed, BFCL, tau^2-bench, long context, strategic reasoning, SVG drawing, video editing and motion design capabilities.
Tests eight RTX Pro 6000 GPUs with 768GB VRAM installed in a Comino Grando chassis to evaluate performance limits.
Compares Qwen3.8-Flash-Next inference via llama.cpp, SGLang, and FreeToken on RTX PRO 6000 Blackwell using AIPerf, measuring first-token latency, decode speed, accuracy on GSM8K/MATH-500, and startup
Benchmarked Qwen3.8-Flash-Next in llama.cpp across 0–96GB VRAM using AMD Ryzen 9 9950X and NVIDIA RTX PRO 6000 Blackwell, measuring decode speeds, prefill gains, and configuration impacts.
Tests combining multiple GPUs into a single desktop setup with 224GB of memory to evaluate performance beyond intended specifications using Strix Halo, LLM and Nvidia components.
Benchmarks DFlash 2, n-gram, MTP and combinations for Qwen 3.8 27B in llama.cpp on RTX PRO 6000 Blackwell using LiveCodeBench and an 18-turn coding session.
Benchmarking DeepSeek V4 Flash 284B with DSpark on RTX PRO 6000 via llama.cpp measures 31 tok/s using multi-turn coding sessions and tests drafter placement in RAM versus VRAM.
Tests MiniMax H3 with SGLang-Diffusion and Turbo LoRA on RTX PRO 6000 Blackwell for t2v, i2v, Ref2VA, and stereo audio, examining Arena rankings and licenses.
Evaluates Laguna S 2.1 118B A8B against its XS version using Performance, Memory, Agency, OpenAI Human Eval, Sand Physics, Kanban, Dungeon Crawler, Blender and Godot tests.
Tests n-gram speculative decoding on Qwen 3.6 27B via llama.cpp against MATH-500, LiveCodeBench, and an 18-prompt iterative coding suite using AMD Ryzen 9 9950X and NVIDIA RTX PRO 6000 Blackwell.
Benchmarking DFlash speculative decoding with Qwen 3.6 27B via llama.cpp on RTX PRO 6000 Blackwell using aiperf and MATH-500, measuring throughput gains and accuracy against baselines.
Tests Tencent Hy3 295B MoE on 2x RTX PRO 6000 Blackwell GPUs using MXFP4 quantization via vLLM and SGLang for coding, agents, and real-world speed.
Deploys DiffusionGemma 26B-A4B in NVFP4 with vLLM on RTX PRO 6000, benchmarking its 256-token parallel generation against standard Gemma 4 to demonstrate a 6.73x speedup.