| Hardware profile | ||||
|---|---|---|---|---|
| Hardware | Memory | Bandwidth | Computing power | vLLM |
| RTX PRO 6000NVIDIA RTX PRO 6000 Blackwell series | 96 GBGPU VRAM | 1792 GB/s | ~3511 TOPS | yes |
| Compatible models Estimated speeds, not benchmark results: calculated from memory bandwidth and model size. Real results can differ significantly because there is no precise formula for deriving LLM generation speed from hardware specifications alone. | |||
|---|---|---|---|
| Model | Size | Approx. Q4 memory | Estimated generation |
| Mistral Medium 3.5Dense | 128B | 76.8 GB | 12 tok/s4-bit quantization |
| Qwen 3.8 Flash NextMoE | 125Bactive 6B | 75 GB | 159 tok/s4-bit quantization |
| Ling 3.0 Flash VLMoE | 124Bactive 5.5B | 74.4 GB | 169 tok/s4-bit quantization |
| Ling 3.0 FlashMoE | 124Bactive 5.1B | 74.4 GB | 179 tok/s4-bit quantization |
| Qwen 3.5 122BMoE | 122Bactive 10B | 73.2 GB | 105 tok/s4-bit quantization |
| Nemotron 3 SuperMoE | 120Bactive 12B | 72 GB | 90 tok/s4-bit quantization |
| Mistral Small 4MoE | 119Bactive 6.5B | 71.4 GB | 149 tok/s4-bit quantization |
| Laguna S 2.1MoE | 118Bactive 8B | 70.8 GB | 126 tok/s4-bit quantization |
| GPT-OSS 120BMoE | 116.8Bactive 5.1B | 70.1 GB | 179 tok/s4-bit quantization |
| Sarvam 105BMoE | 105Bactive 10.3B | 63 GB | 102 tok/s4-bit quantization |
| Qwen 3 Coder Next 80BMoE | 80Bactive 3B | 48 GB | 256 tok/s4-bit quantization |
| Ornith 1.0 35BMoE | 35Bactive 3B | 21 GB | 256 tok/s4-bit quantization |
| Ornith 1.5 35BMoE | 35Bactive 3B | 21 GB | 256 tok/s4-bit quantization |
| Qwen 3.6 35BMoE | 35Bactive 3B | 21 GB | 256 tok/s4-bit quantization |
| Laguna XS 2.1MoE | 33Bactive 3B | 19.8 GB | 256 tok/s4-bit quantization |
| Qwen 2.5 32BDense | 32.5B | 19.5 GB | 48 tok/s4-bit quantization |
| Gemma 4 31BDense | 31B | 18.6 GB | 50 tok/s4-bit quantization |
| Qwen 3 Coder 30BMoE | 30.5Bactive 3.3B | 18.3 GB | 242 tok/s4-bit quantization |
| Granite 4.1 30BDense | 30B | 18 GB | 52 tok/s4-bit quantization |
| Granite 4.2 30BDense | 30B | 18 GB | 52 tok/s4-bit quantization |
| Muse GlimmerDense | 30B | 18 GB | 52 tok/s4-bit quantization |
| GLM 4.7 FlashMoE | 30Bactive 3B | 18 GB | 256 tok/s4-bit quantization |
| Nemotron 3.5 LightningMoE | 30Bactive 3B | 18 GB | 256 tok/s4-bit quantization |
| Fara 1.5 27BDense | 27B | 16.2 GB | 58 tok/s4-bit quantization |
| Qwen 3.6 27BDense | 27B | 16.2 GB | 58 tok/s4-bit quantization |
| Qwen 3.8 27BDense | 27B | 16.2 GB | 58 tok/s4-bit quantization |
| Gemma 4 26BMoE | 26Bactive 4B | 15.6 GB | 213 tok/s4-bit quantization |
| GPT-OSS 20BMoE | 20.9Bactive 3.6B | 12.5 GB | 228 tok/s4-bit quantization |
| Instella-MoE 16B ThinkMoE | 16Bactive 2.8B | 9.6 GB | 267 tok/s4-bit quantization |
| Qwen 3 14BDense | 14B | 8.4 GB | 107 tok/s4-bit quantization |
| Gemma 3 12BDense | 12B | 7.2 GB | 124 tok/s4-bit quantization |
| Gemma 4 12BDense | 12B | 7.2 GB | 124 tok/s4-bit quantization |
| Fara 1.5 9BDense | 9B | 5.4 GB | 160 tok/s4-bit quantization |
| Ornith 1.0 9BDense | 9B | 5.4 GB | 160 tok/s4-bit quantization |
| Ornith 1.5 9BDense | 9B | 5.4 GB | 160 tok/s4-bit quantization |
| Qwen 3.5 9BDense | 9B | 5.4 GB | 160 tok/s4-bit quantization |
| Qwythos 9BDense | 9B | 5.4 GB | 160 tok/s4-bit quantization |
| Granite 4.1 8BDense | 8B | 4.8 GB | 177 tok/s4-bit quantization |
| Granite 4.2 8BDense | 8B | 4.8 GB | 177 tok/s4-bit quantization |
| Qwen 3 8BDense | 8B | 4.8 GB | 177 tok/s4-bit quantization |
| Ling 3.0 TinyMoE | 7.9Bactive 1.3B | 4.7 GB | 394 tok/s4-bit quantization |
| Fara 1.5 4BDense | 4B | 2.4 GB | 313 tok/s4-bit quantization |
| Gemma 3 4BDense | 4B | 2.4 GB | 313 tok/s4-bit quantization |
| Qwen 3 4B Instruct 2507Dense | 4B | 2.4 GB | 313 tok/s4-bit quantization |
| Qwen 3.5 4BDense | 4B | 2.4 GB | 313 tok/s4-bit quantization |
| Spark X2.5 4BDense | 4B | 2.4 GB | 313 tok/s4-bit quantization |
| Granite 4.1 3BDense | 3B | 1.8 GB | 388 tok/s4-bit quantization |
| Granite 4.2 3BDense | 3B | 1.8 GB | 388 tok/s4-bit quantization |
| Llama 3.2 3B InstructDense | 3B | 1.8 GB | 388 tok/s4-bit quantization |
| Ministral 3 3BDense | 3B | 1.8 GB | 388 tok/s4-bit quantization |
| Nanbeige 4.2 3BDense | 3B | 1.8 GB | 388 tok/s4-bit quantization |
Tests Qwen3.8-Flash-Next on RTX PRO 6000 with SGLang, measuring speed, BFCL, tau^2-bench, long context, strategic reasoning, SVG drawing, video editing and motion design capabilities.
Tests eight RTX Pro 6000 GPUs with 768GB VRAM installed in a Comino Grando chassis to evaluate performance limits.
Compares Qwen3.8-Flash-Next inference via llama.cpp, SGLang, and FreeToken on RTX PRO 6000 Blackwell using AIPerf, measuring first-token latency, decode speed, accuracy on GSM8K/MATH-500, and startup
Benchmarked Qwen3.8-Flash-Next in llama.cpp across 0–96GB VRAM using AMD Ryzen 9 9950X and NVIDIA RTX PRO 6000 Blackwell, measuring decode speeds, prefill gains, and configuration impacts.
Tests combining multiple GPUs into a single desktop setup with 224GB of memory to evaluate performance beyond intended specifications using Strix Halo, LLM and Nvidia components.
Benchmarks DFlash 2, n-gram, MTP and combinations for Qwen 3.8 27B in llama.cpp on RTX PRO 6000 Blackwell using LiveCodeBench and an 18-turn coding session.
Benchmarking DeepSeek V4 Flash 284B with DSpark on RTX PRO 6000 via llama.cpp measures 31 tok/s using multi-turn coding sessions and tests drafter placement in RAM versus VRAM.
Tests MiniMax H3 with SGLang-Diffusion and Turbo LoRA on RTX PRO 6000 Blackwell for t2v, i2v, Ref2VA, and stereo audio, examining Arena rankings and licenses.
Evaluates Laguna S 2.1 118B A8B against its XS version using Performance, Memory, Agency, OpenAI Human Eval, Sand Physics, Kanban, Dungeon Crawler, Blender and Godot tests.
Tests n-gram speculative decoding on Qwen 3.6 27B via llama.cpp against MATH-500, LiveCodeBench, and an 18-prompt iterative coding suite using AMD Ryzen 9 9950X and NVIDIA RTX PRO 6000 Blackwell.
Benchmarking DFlash speculative decoding with Qwen 3.6 27B via llama.cpp on RTX PRO 6000 Blackwell using aiperf and MATH-500, measuring throughput gains and accuracy against baselines.
Tests Tencent Hy3 295B MoE on 2x RTX PRO 6000 Blackwell GPUs using MXFP4 quantization via vLLM and SGLang for coding, agents, and real-world speed.
Deploys DiffusionGemma 26B-A4B in NVFP4 with vLLM on RTX PRO 6000, benchmarking its 256-token parallel generation against standard Gemma 4 to demonstrate a 6.73x speedup.