Hardware profile
Hardware	Memory	Bandwidth	Computing power	vLLM
GB200 NVL72NVIDIA rack-scale liquid-cooled system with 72 Blackwell GPUs, 36 Grace CPUs, 13.4 TB total HBM3e, 576 TB/s aggregate HBM bandwidth and 130 TB/s aggregate NVLink	13400 GBGPU HBM3e	8000 GB/s	~20000 TOPS	yes

Compatible models Estimated speeds, not benchmark results: calculated from memory bandwidth and model size. Real results can differ significantly because there is no precise formula for deriving LLM generation speed from hardware specifications alone.
Model	Size	Approx. Q4 memory	Estimated generation
Kimi K3MoE	2800Bactive 104B	1680 GB	49 tok/s4-bit quantization
Qwen 3.8 2.4TMoE	2400Bactive 95B	1440 GB	54 tok/s4-bit quantization
DeepSeek V4 ProMoE	1600Bactive 49B	960 GB	97 tok/s4-bit quantization
MiMo V2.5 ProMoE	1020Bactive 42B	612 GB	111 tok/s4-bit quantization
Kimi K2.6MoE	1000Bactive 32B	600 GB	138 tok/s4-bit quantization
Kimi K2.7MoE	1000Bactive 32B	600 GB	138 tok/s4-bit quantization
Hy4MoE	770Bactive 49B	462 GB	97 tok/s4-bit quantization
GLM 5.1MoE	744Bactive 40B	446.4 GB	115 tok/s4-bit quantization
GLM 5.2MoE	744Bactive 40B	446.4 GB	115 tok/s4-bit quantization
DeepSeek V4.1 FlashMoE	552Bactive 16B	331.2 GB	229 tok/s4-bit quantization
Nemotron 3 UltraMoE	550Bactive 55B	330 GB	88 tok/s4-bit quantization
Qwen 3 Coder 480BMoE	480Bactive 35B	288 GB	129 tok/s4-bit quantization
MiniMax M3MoE	428Bactive 23B	256.8 GB	178 tok/s4-bit quantization
Ornith 1.5 397BMoE	397Bactive 17B	238.2 GB	220 tok/s4-bit quantization
Qwen 3.5 397BMoE	397Bactive 17B	238.2 GB	220 tok/s4-bit quantization
GLM 5.3 FlashMoE	320Bactive 18B	192 GB	212 tok/s4-bit quantization
MiMo V2.5MoE	310Bactive 15B	186 GB	239 tok/s4-bit quantization
Hy3MoE	295Bactive 21B	177 GB	190 tok/s4-bit quantization
DeepSeek V4 FlashMoE	284Bactive 13B	170.4 GB	261 tok/s4-bit quantization
Inkling SmallMoE	276Bactive 12B	165.6 GB	274 tok/s4-bit quantization
Solar Open 2MoE	250Bactive 15B	150 GB	239 tok/s4-bit quantization
Qwen 3 235BMoE	235Bactive 22B	141 GB	184 tok/s4-bit quantization
Laguna M.1MoE	225Bactive 23B	135 GB	178 tok/s4-bit quantization
Command A+MoE	218Bactive 25B	130.8 GB	167 tok/s4-bit quantization
Step 3.7 FlashMoE	198Bactive 11B	118.8 GB	288 tok/s4-bit quantization
Mistral Medium 3.5Dense	128B	76.8 GB	54 tok/s4-bit quantization
Qwen 3.8 Flash NextMoE	125Bactive 6B	75 GB	388 tok/s4-bit quantization
Ling 3.0 Flash VLMoE	124Bactive 5.5B	74.4 GB	402 tok/s4-bit quantization
Ling 3.0 FlashMoE	124Bactive 5.1B	74.4 GB	414 tok/s4-bit quantization
Qwen 3.5 122BMoE	122Bactive 10B	73.2 GB	304 tok/s4-bit quantization
Nemotron 3 SuperMoE	120Bactive 12B	72 GB	274 tok/s4-bit quantization
Mistral Small 4MoE	119Bactive 6.5B	71.4 GB	375 tok/s4-bit quantization
Laguna S 2.1MoE	118Bactive 8B	70.8 GB	341 tok/s4-bit quantization
GPT-OSS 120BMoE	116.8Bactive 5.1B	70.1 GB	414 tok/s4-bit quantization
Sarvam 105BMoE	105Bactive 10.3B	63 GB	299 tok/s4-bit quantization
Qwen 3 Coder Next 80BMoE	80Bactive 3B	48 GB	491 tok/s4-bit quantization
Ornith 1.0 35BMoE	35Bactive 3B	21 GB	491 tok/s4-bit quantization
Ornith 1.5 35BMoE	35Bactive 3B	21 GB	491 tok/s4-bit quantization
Qwen 3.6 35BMoE	35Bactive 3B	21 GB	491 tok/s4-bit quantization
Laguna XS 2.1MoE	33Bactive 3B	19.8 GB	491 tok/s4-bit quantization
Qwen 2.5 32BDense	32.5B	19.5 GB	192 tok/s4-bit quantization
Gemma 4 31BDense	31B	18.6 GB	200 tok/s4-bit quantization
Qwen 3 Coder 30BMoE	30.5Bactive 3.3B	18.3 GB	478 tok/s4-bit quantization
Granite 4.1 30BDense	30B	18 GB	206 tok/s4-bit quantization
Granite 4.2 30BDense	30B	18 GB	206 tok/s4-bit quantization
Muse GlimmerDense	30B	18 GB	206 tok/s4-bit quantization
GLM 4.7 FlashMoE	30Bactive 3B	18 GB	491 tok/s4-bit quantization
Nemotron 3.5 LightningMoE	30Bactive 3B	18 GB	491 tok/s4-bit quantization
Fara 1.5 27BDense	27B	16.2 GB	225 tok/s4-bit quantization
Qwen 3.6 27BDense	27B	16.2 GB	225 tok/s4-bit quantization
Qwen 3.8 27BDense	27B	16.2 GB	225 tok/s4-bit quantization
Gemma 4 26BMoE	26Bactive 4B	15.6 GB	451 tok/s4-bit quantization
GPT-OSS 20BMoE	20.9Bactive 3.6B	12.5 GB	466 tok/s4-bit quantization
Instella-MoE 16B ThinkMoE	16Bactive 2.8B	9.6 GB	500 tok/s4-bit quantization
Qwen 3 14BDense	14B	8.4 GB	376 tok/s4-bit quantization
Gemma 3 12BDense	12B	7.2 GB	419 tok/s4-bit quantization
Gemma 4 12BDense	12B	7.2 GB	419 tok/s4-bit quantization
Fara 1.5 9BDense	9B	5.4 GB	505 tok/s4-bit quantization
Ornith 1.0 9BDense	9B	5.4 GB	505 tok/s4-bit quantization
Ornith 1.5 9BDense	9B	5.4 GB	505 tok/s4-bit quantization
Qwen 3.5 9BDense	9B	5.4 GB	505 tok/s4-bit quantization
Qwythos 9BDense	9B	5.4 GB	505 tok/s4-bit quantization
Granite 4.1 8BDense	8B	4.8 GB	543 tok/s4-bit quantization
Granite 4.2 8BDense	8B	4.8 GB	543 tok/s4-bit quantization
Qwen 3 8BDense	8B	4.8 GB	543 tok/s4-bit quantization
Ling 3.0 TinyMoE	7.9Bactive 1.3B	4.7 GB	577 tok/s4-bit quantization
Fara 1.5 4BDense	4B	2.4 GB	771 tok/s4-bit quantization
Gemma 3 4BDense	4B	2.4 GB	771 tok/s4-bit quantization
Qwen 3 4B Instruct 2507Dense	4B	2.4 GB	771 tok/s4-bit quantization
Qwen 3.5 4BDense	4B	2.4 GB	771 tok/s4-bit quantization
Spark X2.5 4BDense	4B	2.4 GB	771 tok/s4-bit quantization
Granite 4.1 3BDense	3B	1.8 GB	862 tok/s4-bit quantization
Granite 4.2 3BDense	3B	1.8 GB	862 tok/s4-bit quantization
Llama 3.2 3B InstructDense	3B	1.8 GB	862 tok/s4-bit quantization
Ministral 3 3BDense	3B	1.8 GB	862 tok/s4-bit quantization
Nanbeige 4.2 3BDense	3B	1.8 GB	862 tok/s4-bit quantization