Local AI hardware: DGX Station

Hardware profile
Hardware	Memory	Bandwidth	Computing power	vLLM
DGX StationNVIDIA DGX Station with GB300 Grace Blackwell Ultra	748 GBCoherent Memory	7100 GB/s	~20000 TOPS	yes

Compatible models Estimated speeds, not benchmark results: calculated from memory bandwidth and model size. Real results can differ significantly because there is no precise formula for deriving LLM generation speed from hardware specifications alone.
Model	Size	Approx. Q4 memory	Estimated generation
MiMo V2.5 ProMoE	1020Bactive 42B	612 GB	100 tok/s4-bit quantization
Kimi K2.6MoE	1000Bactive 32B	600 GB	125 tok/s4-bit quantization
Kimi K2.7MoE	1000Bactive 32B	600 GB	125 tok/s4-bit quantization
Hy4MoE	770Bactive 49B	462 GB	88 tok/s4-bit quantization
GLM 5.1MoE	744Bactive 40B	446.4 GB	104 tok/s4-bit quantization
GLM 5.2MoE	744Bactive 40B	446.4 GB	104 tok/s4-bit quantization
DeepSeek V4.1 FlashMoE	552Bactive 16B	331.2 GB	211 tok/s4-bit quantization
Nemotron 3 UltraMoE	550Bactive 55B	330 GB	79 tok/s4-bit quantization
Qwen 3 Coder 480BMoE	480Bactive 35B	288 GB	117 tok/s4-bit quantization
MiniMax M3MoE	428Bactive 23B	256.8 GB	163 tok/s4-bit quantization
Ornith 1.5 397BMoE	397Bactive 17B	238.2 GB	203 tok/s4-bit quantization
Qwen 3.5 397BMoE	397Bactive 17B	238.2 GB	203 tok/s4-bit quantization
GLM 5.3 FlashMoE	320Bactive 18B	192 GB	195 tok/s4-bit quantization
MiMo V2.5MoE	310Bactive 15B	186 GB	221 tok/s4-bit quantization
Hy3MoE	295Bactive 21B	177 GB	174 tok/s4-bit quantization
DeepSeek V4 FlashMoE	284Bactive 13B	170.4 GB	242 tok/s4-bit quantization
Inkling SmallMoE	276Bactive 12B	165.6 GB	255 tok/s4-bit quantization
Solar Open 2MoE	250Bactive 15B	150 GB	221 tok/s4-bit quantization
Qwen 3 235BMoE	235Bactive 22B	141 GB	168 tok/s4-bit quantization
Laguna M.1MoE	225Bactive 23B	135 GB	163 tok/s4-bit quantization
Command A+MoE	218Bactive 25B	130.8 GB	153 tok/s4-bit quantization
Step 3.7 FlashMoE	198Bactive 11B	118.8 GB	269 tok/s4-bit quantization
Mistral Medium 3.5Dense	128B	76.8 GB	48 tok/s4-bit quantization
Qwen 3.8 Flash NextMoE	125Bactive 6B	75 GB	369 tok/s4-bit quantization
Ling 3.0 Flash VLMoE	124Bactive 5.5B	74.4 GB	383 tok/s4-bit quantization
Ling 3.0 FlashMoE	124Bactive 5.1B	74.4 GB	395 tok/s4-bit quantization
Qwen 3.5 122BMoE	122Bactive 10B	73.2 GB	284 tok/s4-bit quantization
Nemotron 3 SuperMoE	120Bactive 12B	72 GB	255 tok/s4-bit quantization
Mistral Small 4MoE	119Bactive 6.5B	71.4 GB	356 tok/s4-bit quantization
Laguna S 2.1MoE	118Bactive 8B	70.8 GB	321 tok/s4-bit quantization
GPT-OSS 120BMoE	116.8Bactive 5.1B	70.1 GB	395 tok/s4-bit quantization
Sarvam 105BMoE	105Bactive 10.3B	63 GB	279 tok/s4-bit quantization
Qwen 3 Coder Next 80BMoE	80Bactive 3B	48 GB	475 tok/s4-bit quantization
Ornith 1.0 35BMoE	35Bactive 3B	21 GB	475 tok/s4-bit quantization
Ornith 1.5 35BMoE	35Bactive 3B	21 GB	475 tok/s4-bit quantization
Qwen 3.6 35BMoE	35Bactive 3B	21 GB	475 tok/s4-bit quantization
Laguna XS 2.1MoE	33Bactive 3B	19.8 GB	475 tok/s4-bit quantization
Qwen 2.5 32BDense	32.5B	19.5 GB	174 tok/s4-bit quantization
Gemma 4 31BDense	31B	18.6 GB	181 tok/s4-bit quantization
Qwen 3 Coder 30BMoE	30.5Bactive 3.3B	18.3 GB	462 tok/s4-bit quantization
Granite 4.1 30BDense	30B	18 GB	186 tok/s4-bit quantization
Granite 4.2 30BDense	30B	18 GB	186 tok/s4-bit quantization
Muse GlimmerDense	30B	18 GB	186 tok/s4-bit quantization
GLM 4.7 FlashMoE	30Bactive 3B	18 GB	475 tok/s4-bit quantization
Nemotron 3.5 LightningMoE	30Bactive 3B	18 GB	475 tok/s4-bit quantization
Fara 1.5 27BDense	27B	16.2 GB	204 tok/s4-bit quantization
Qwen 3.6 27BDense	27B	16.2 GB	204 tok/s4-bit quantization
Qwen 3.8 27BDense	27B	16.2 GB	204 tok/s4-bit quantization
Gemma 4 26BMoE	26Bactive 4B	15.6 GB	433 tok/s4-bit quantization
GPT-OSS 20BMoE	20.9Bactive 3.6B	12.5 GB	449 tok/s4-bit quantization
Instella-MoE 16B ThinkMoE	16Bactive 2.8B	9.6 GB	484 tok/s4-bit quantization
Qwen 3 14BDense	14B	8.4 GB	344 tok/s4-bit quantization
Gemma 3 12BDense	12B	7.2 GB	385 tok/s4-bit quantization
Gemma 4 12BDense	12B	7.2 GB	385 tok/s4-bit quantization
Fara 1.5 9BDense	9B	5.4 GB	468 tok/s4-bit quantization
Ornith 1.0 9BDense	9B	5.4 GB	468 tok/s4-bit quantization
Ornith 1.5 9BDense	9B	5.4 GB	468 tok/s4-bit quantization
Qwen 3.5 9BDense	9B	5.4 GB	468 tok/s4-bit quantization
Qwythos 9BDense	9B	5.4 GB	468 tok/s4-bit quantization
Granite 4.1 8BDense	8B	4.8 GB	505 tok/s4-bit quantization
Granite 4.2 8BDense	8B	4.8 GB	505 tok/s4-bit quantization
Qwen 3 8BDense	8B	4.8 GB	505 tok/s4-bit quantization
Ling 3.0 TinyMoE	7.9Bactive 1.3B	4.7 GB	567 tok/s4-bit quantization
Fara 1.5 4BDense	4B	2.4 GB	732 tok/s4-bit quantization
Gemma 3 4BDense	4B	2.4 GB	732 tok/s4-bit quantization
Qwen 3 4B Instruct 2507Dense	4B	2.4 GB	732 tok/s4-bit quantization
Qwen 3.5 4BDense	4B	2.4 GB	732 tok/s4-bit quantization
Spark X2.5 4BDense	4B	2.4 GB	732 tok/s4-bit quantization
Granite 4.1 3BDense	3B	1.8 GB	825 tok/s4-bit quantization
Granite 4.2 3BDense	3B	1.8 GB	825 tok/s4-bit quantization
Llama 3.2 3B InstructDense	3B	1.8 GB	825 tok/s4-bit quantization
Ministral 3 3BDense	3B	1.8 GB	825 tok/s4-bit quantization
Nanbeige 4.2 3BDense	3B	1.8 GB	825 tok/s4-bit quantization

1 video found Showing 1-1

Video Alex Ziskind 2026-08-01

DGX Station running GLM5.2 in VSCode

Demonstrates a single-user DGX Station setup with 748GB of unified memory running GLM 5.2 in VSCode.

GLM 5.2 DGX Station