Hardware profile
Hardware	Memory	Bandwidth	Computing power	vLLM
DGX H200NVIDIA 8-GPU Hopper rackmount AI system with 1,128 GB total HBM3e, 141 GB per GPU and 4.8 TB/s HBM bandwidth per GPU	1128 GBGPU HBM3e	4800 GB/s	~3958 TOPS	yes

Compatible models Estimated speeds, not benchmark results: calculated from memory bandwidth and model size. Real results can differ significantly because there is no precise formula for deriving LLM generation speed from hardware specifications alone.
Model	Size	Approx. Q4 memory	Estimated generation
DeepSeek V4 ProMoE	1600Bactive 49B	960 GB	62 tok/s4-bit quantization
MiMo V2.5 ProMoE	1020Bactive 42B	612 GB	71 tok/s4-bit quantization
Kimi K2.6MoE	1000Bactive 32B	600 GB	90 tok/s4-bit quantization
Kimi K2.7MoE	1000Bactive 32B	600 GB	90 tok/s4-bit quantization
Hy4MoE	770Bactive 49B	462 GB	62 tok/s4-bit quantization
GLM 5.1MoE	744Bactive 40B	446.4 GB	74 tok/s4-bit quantization
GLM 5.2MoE	744Bactive 40B	446.4 GB	74 tok/s4-bit quantization
DeepSeek V4.1 FlashMoE	552Bactive 16B	331.2 GB	159 tok/s4-bit quantization
Nemotron 3 UltraMoE	550Bactive 55B	330 GB	55 tok/s4-bit quantization
Qwen 3 Coder 480BMoE	480Bactive 35B	288 GB	83 tok/s4-bit quantization
MiniMax M3MoE	428Bactive 23B	256.8 GB	119 tok/s4-bit quantization
Ornith 1.5 397BMoE	397Bactive 17B	238.2 GB	152 tok/s4-bit quantization
Qwen 3.5 397BMoE	397Bactive 17B	238.2 GB	152 tok/s4-bit quantization
GLM 5.3 FlashMoE	320Bactive 18B	192 GB	145 tok/s4-bit quantization
MiMo V2.5MoE	310Bactive 15B	186 GB	167 tok/s4-bit quantization
Hy3MoE	295Bactive 21B	177 GB	129 tok/s4-bit quantization
DeepSeek V4 FlashMoE	284Bactive 13B	170.4 GB	186 tok/s4-bit quantization
Inkling SmallMoE	276Bactive 12B	165.6 GB	197 tok/s4-bit quantization
Solar Open 2MoE	250Bactive 15B	150 GB	167 tok/s4-bit quantization
Qwen 3 235BMoE	235Bactive 22B	141 GB	124 tok/s4-bit quantization
Laguna M.1MoE	225Bactive 23B	135 GB	119 tok/s4-bit quantization
Command A+MoE	218Bactive 25B	130.8 GB	111 tok/s4-bit quantization
Step 3.7 FlashMoE	198Bactive 11B	118.8 GB	209 tok/s4-bit quantization
Mistral Medium 3.5Dense	128B	76.8 GB	33 tok/s4-bit quantization
Qwen 3.8 Flash NextMoE	125Bactive 6B	75 GB	304 tok/s4-bit quantization
Ling 3.0 Flash VLMoE	124Bactive 5.5B	74.4 GB	318 tok/s4-bit quantization
Ling 3.0 FlashMoE	124Bactive 5.1B	74.4 GB	331 tok/s4-bit quantization
Qwen 3.5 122BMoE	122Bactive 10B	73.2 GB	223 tok/s4-bit quantization
Nemotron 3 SuperMoE	120Bactive 12B	72 GB	197 tok/s4-bit quantization
Mistral Small 4MoE	119Bactive 6.5B	71.4 GB	291 tok/s4-bit quantization
Laguna S 2.1MoE	118Bactive 8B	70.8 GB	257 tok/s4-bit quantization
GPT-OSS 120BMoE	116.8Bactive 5.1B	70.1 GB	331 tok/s4-bit quantization
Sarvam 105BMoE	105Bactive 10.3B	63 GB	219 tok/s4-bit quantization
Qwen 3 Coder Next 80BMoE	80Bactive 3B	48 GB	417 tok/s4-bit quantization
Ornith 1.0 35BMoE	35Bactive 3B	21 GB	417 tok/s4-bit quantization
Ornith 1.5 35BMoE	35Bactive 3B	21 GB	417 tok/s4-bit quantization
Qwen 3.6 35BMoE	35Bactive 3B	21 GB	417 tok/s4-bit quantization
Laguna XS 2.1MoE	33Bactive 3B	19.8 GB	417 tok/s4-bit quantization
Qwen 2.5 32BDense	32.5B	19.5 GB	122 tok/s4-bit quantization
Gemma 4 31BDense	31B	18.6 GB	128 tok/s4-bit quantization
Qwen 3 Coder 30BMoE	30.5Bactive 3.3B	18.3 GB	402 tok/s4-bit quantization
Granite 4.1 30BDense	30B	18 GB	132 tok/s4-bit quantization
Granite 4.2 30BDense	30B	18 GB	132 tok/s4-bit quantization
Muse GlimmerDense	30B	18 GB	132 tok/s4-bit quantization
GLM 4.7 FlashMoE	30Bactive 3B	18 GB	417 tok/s4-bit quantization
Nemotron 3.5 LightningMoE	30Bactive 3B	18 GB	417 tok/s4-bit quantization
Fara 1.5 27BDense	27B	16.2 GB	145 tok/s4-bit quantization
Qwen 3.6 27BDense	27B	16.2 GB	145 tok/s4-bit quantization
Qwen 3.8 27BDense	27B	16.2 GB	145 tok/s4-bit quantization
Gemma 4 26BMoE	26Bactive 4B	15.6 GB	371 tok/s4-bit quantization
GPT-OSS 20BMoE	20.9Bactive 3.6B	12.5 GB	388 tok/s4-bit quantization
Instella-MoE 16B ThinkMoE	16Bactive 2.8B	9.6 GB	428 tok/s4-bit quantization
Qwen 3 14BDense	14B	8.4 GB	254 tok/s4-bit quantization
Gemma 3 12BDense	12B	7.2 GB	287 tok/s4-bit quantization
Gemma 4 12BDense	12B	7.2 GB	287 tok/s4-bit quantization
Fara 1.5 9BDense	9B	5.4 GB	357 tok/s4-bit quantization
Ornith 1.0 9BDense	9B	5.4 GB	357 tok/s4-bit quantization
Ornith 1.5 9BDense	9B	5.4 GB	357 tok/s4-bit quantization
Qwen 3.5 9BDense	9B	5.4 GB	357 tok/s4-bit quantization
Qwythos 9BDense	9B	5.4 GB	357 tok/s4-bit quantization
Granite 4.1 8BDense	8B	4.8 GB	389 tok/s4-bit quantization
Granite 4.2 8BDense	8B	4.8 GB	389 tok/s4-bit quantization
Qwen 3 8BDense	8B	4.8 GB	389 tok/s4-bit quantization
Ling 3.0 TinyMoE	7.9Bactive 1.3B	4.7 GB	529 tok/s4-bit quantization
Fara 1.5 4BDense	4B	2.4 GB	602 tok/s4-bit quantization
Gemma 3 4BDense	4B	2.4 GB	602 tok/s4-bit quantization
Qwen 3 4B Instruct 2507Dense	4B	2.4 GB	602 tok/s4-bit quantization
Qwen 3.5 4BDense	4B	2.4 GB	602 tok/s4-bit quantization
Spark X2.5 4BDense	4B	2.4 GB	602 tok/s4-bit quantization
Granite 4.1 3BDense	3B	1.8 GB	698 tok/s4-bit quantization
Granite 4.2 3BDense	3B	1.8 GB	698 tok/s4-bit quantization
Llama 3.2 3B InstructDense	3B	1.8 GB	698 tok/s4-bit quantization
Ministral 3 3BDense	3B	1.8 GB	698 tok/s4-bit quantization
Nanbeige 4.2 3BDense	3B	1.8 GB	698 tok/s4-bit quantization