Hardware profile
Hardware	Memory	Bandwidth	Computing power	vLLM
M5 UltraApple Mac Studio M5 Ultra 80-core GPU configuration with 256 GB unified	256 GBUnified RAM	1200 GB/s	~540 TOPS	no

Compatible models Estimated speeds, not benchmark results: calculated from memory bandwidth and model size. Real results can differ significantly because there is no precise formula for deriving LLM generation speed from hardware specifications alone.
Model	Size	Approx. Q4 memory	Estimated generation
Ornith 1.5 397BMoE	397Bactive 17B	238.2 GB	46 tok/s4-bit quantization
Qwen 3.5 397BMoE	397Bactive 17B	238.2 GB	46 tok/s4-bit quantization
GLM 5.3 FlashMoE	320Bactive 18B	192 GB	43 tok/s4-bit quantization
MiMo V2.5MoE	310Bactive 15B	186 GB	51 tok/s4-bit quantization
Hy3MoE	295Bactive 21B	177 GB	37 tok/s4-bit quantization
DeepSeek V4 FlashMoE	284Bactive 13B	170.4 GB	58 tok/s4-bit quantization
Inkling SmallMoE	276Bactive 12B	165.6 GB	63 tok/s4-bit quantization
Solar Open 2MoE	250Bactive 15B	150 GB	51 tok/s4-bit quantization
Qwen 3 235BMoE	235Bactive 22B	141 GB	36 tok/s4-bit quantization
Laguna M.1MoE	225Bactive 23B	135 GB	34 tok/s4-bit quantization
Command A+MoE	218Bactive 25B	130.8 GB	31 tok/s4-bit quantization
Step 3.7 FlashMoE	198Bactive 11B	118.8 GB	68 tok/s4-bit quantization
Mistral Medium 3.5Dense	128B	76.8 GB	8 tok/s4-bit quantization
Qwen 3.8 Flash NextMoE	125Bactive 6B	75 GB	115 tok/s4-bit quantization
Ling 3.0 Flash VLMoE	124Bactive 5.5B	74.4 GB	124 tok/s4-bit quantization
Ling 3.0 FlashMoE	124Bactive 5.1B	74.4 GB	132 tok/s4-bit quantization
Qwen 3.5 122BMoE	122Bactive 10B	73.2 GB	74 tok/s4-bit quantization
Nemotron 3 SuperMoE	120Bactive 12B	72 GB	63 tok/s4-bit quantization
Mistral Small 4MoE	119Bactive 6.5B	71.4 GB	108 tok/s4-bit quantization
Laguna S 2.1MoE	118Bactive 8B	70.8 GB	90 tok/s4-bit quantization
GPT-OSS 120BMoE	116.8Bactive 5.1B	70.1 GB	132 tok/s4-bit quantization
Sarvam 105BMoE	105Bactive 10.3B	63 GB	72 tok/s4-bit quantization
Qwen 3 Coder Next 80BMoE	80Bactive 3B	48 GB	197 tok/s4-bit quantization
Ornith 1.0 35BMoE	35Bactive 3B	21 GB	197 tok/s4-bit quantization
Ornith 1.5 35BMoE	35Bactive 3B	21 GB	197 tok/s4-bit quantization
Qwen 3.6 35BMoE	35Bactive 3B	21 GB	197 tok/s4-bit quantization
Laguna XS 2.1MoE	33Bactive 3B	19.8 GB	197 tok/s4-bit quantization
Qwen 2.5 32BDense	32.5B	19.5 GB	33 tok/s4-bit quantization
Gemma 4 31BDense	31B	18.6 GB	34 tok/s4-bit quantization
Qwen 3 Coder 30BMoE	30.5Bactive 3.3B	18.3 GB	184 tok/s4-bit quantization
Granite 4.1 30BDense	30B	18 GB	35 tok/s4-bit quantization
Granite 4.2 30BDense	30B	18 GB	35 tok/s4-bit quantization
Muse GlimmerDense	30B	18 GB	35 tok/s4-bit quantization
GLM 4.7 FlashMoE	30Bactive 3B	18 GB	197 tok/s4-bit quantization
Nemotron 3.5 LightningMoE	30Bactive 3B	18 GB	197 tok/s4-bit quantization
Fara 1.5 27BDense	27B	16.2 GB	39 tok/s4-bit quantization
Qwen 3.6 27BDense	27B	16.2 GB	39 tok/s4-bit quantization
Qwen 3.8 27BDense	27B	16.2 GB	39 tok/s4-bit quantization
Gemma 4 26BMoE	26Bactive 4B	15.6 GB	159 tok/s4-bit quantization
GPT-OSS 20BMoE	20.9Bactive 3.6B	12.5 GB	172 tok/s4-bit quantization
Instella-MoE 16B ThinkMoE	16Bactive 2.8B	9.6 GB	206 tok/s4-bit quantization
Qwen 3 14BDense	14B	8.4 GB	74 tok/s4-bit quantization
Gemma 3 12BDense	12B	7.2 GB	85 tok/s4-bit quantization
Gemma 4 12BDense	12B	7.2 GB	85 tok/s4-bit quantization
Fara 1.5 9BDense	9B	5.4 GB	111 tok/s4-bit quantization
Ornith 1.0 9BDense	9B	5.4 GB	111 tok/s4-bit quantization
Ornith 1.5 9BDense	9B	5.4 GB	111 tok/s4-bit quantization
Qwen 3.5 9BDense	9B	5.4 GB	111 tok/s4-bit quantization
Qwythos 9BDense	9B	5.4 GB	111 tok/s4-bit quantization
Granite 4.1 8BDense	8B	4.8 GB	124 tok/s4-bit quantization
Granite 4.2 8BDense	8B	4.8 GB	124 tok/s4-bit quantization
Qwen 3 8BDense	8B	4.8 GB	124 tok/s4-bit quantization
Ling 3.0 TinyMoE	7.9Bactive 1.3B	4.7 GB	328 tok/s4-bit quantization
Fara 1.5 4BDense	4B	2.4 GB	227 tok/s4-bit quantization
Gemma 3 4BDense	4B	2.4 GB	227 tok/s4-bit quantization
Qwen 3 4B Instruct 2507Dense	4B	2.4 GB	227 tok/s4-bit quantization
Qwen 3.5 4BDense	4B	2.4 GB	227 tok/s4-bit quantization
Spark X2.5 4BDense	4B	2.4 GB	227 tok/s4-bit quantization
Granite 4.1 3BDense	3B	1.8 GB	287 tok/s4-bit quantization
Granite 4.2 3BDense	3B	1.8 GB	287 tok/s4-bit quantization
Llama 3.2 3B InstructDense	3B	1.8 GB	287 tok/s4-bit quantization
Ministral 3 3BDense	3B	1.8 GB	287 tok/s4-bit quantization
Nanbeige 4.2 3BDense	3B	1.8 GB	287 tok/s4-bit quantization