| Hardware profile | ||||
|---|---|---|---|---|
| Hardware | Memory | Bandwidth | Computing power | vLLM |
| Mac mini M4Apple Mac mini M4 system with 24 GB unified memory | 24 GBUnified RAM | 120 GB/s | ~40 TOPS | no |
| Compatible models Estimated speeds, not benchmark results: calculated from memory bandwidth and model size. Real results can differ significantly because there is no precise formula for deriving LLM generation speed from hardware specifications alone. | |||
|---|---|---|---|
| Model | Size | Approx. Q4 memory | Estimated generation |
| Ornith 1.0 35BMoE | 35Bactive 3B | 21 GB | 26 tok/s4-bit quantization |
| Ornith 1.5 35BMoE | 35Bactive 3B | 21 GB | 26 tok/s4-bit quantization |
| Qwen 3.6 35BMoE | 35Bactive 3B | 21 GB | 26 tok/s4-bit quantization |
| Laguna XS 2.1MoE | 33Bactive 3B | 19.8 GB | 26 tok/s4-bit quantization |
| Qwen 2.5 32BDense | 32.5B | 19.5 GB | 3 tok/s4-bit quantization |
| Gemma 4 31BDense | 31B | 18.6 GB | 3 tok/s4-bit quantization |
| Qwen 3 Coder 30BMoE | 30.5Bactive 3.3B | 18.3 GB | 24 tok/s4-bit quantization |
| Granite 4.1 30BDense | 30B | 18 GB | 3 tok/s4-bit quantization |
| Granite 4.2 30BDense | 30B | 18 GB | 3 tok/s4-bit quantization |
| Muse GlimmerDense | 30B | 18 GB | 3 tok/s4-bit quantization |
| GLM 4.7 FlashMoE | 30Bactive 3B | 18 GB | 26 tok/s4-bit quantization |
| Nemotron 3.5 LightningMoE | 30Bactive 3B | 18 GB | 26 tok/s4-bit quantization |
| Fara 1.5 27BDense | 27B | 16.2 GB | 4 tok/s4-bit quantization |
| Qwen 3.6 27BDense | 27B | 16.2 GB | 4 tok/s4-bit quantization |
| Qwen 3.8 27BDense | 27B | 16.2 GB | 4 tok/s4-bit quantization |
| Gemma 4 26BMoE | 26Bactive 4B | 15.6 GB | 20 tok/s4-bit quantization |
| GPT-OSS 20BMoE | 20.9Bactive 3.6B | 12.5 GB | 22 tok/s4-bit quantization |
| Instella-MoE 16B ThinkMoE | 16Bactive 2.8B | 9.6 GB | 28 tok/s4-bit quantization |
| Qwen 3 14BDense | 14B | 8.4 GB | 7 tok/s4-bit quantization |
| Gemma 3 12BDense | 12B | 7.2 GB | 9 tok/s4-bit quantization |
| Gemma 4 12BDense | 12B | 7.2 GB | 9 tok/s4-bit quantization |
| Fara 1.5 9BDense | 9B | 5.4 GB | 12 tok/s4-bit quantization |
| Ornith 1.0 9BDense | 9B | 5.4 GB | 12 tok/s4-bit quantization |
| Ornith 1.5 9BDense | 9B | 5.4 GB | 12 tok/s4-bit quantization |
| Qwen 3.5 9BDense | 9B | 5.4 GB | 12 tok/s4-bit quantization |
| Qwythos 9BDense | 9B | 5.4 GB | 12 tok/s4-bit quantization |
| Granite 4.1 8BDense | 8B | 4.8 GB | 13 tok/s4-bit quantization |
| Granite 4.2 8BDense | 8B | 4.8 GB | 13 tok/s4-bit quantization |
| Qwen 3 8BDense | 8B | 4.8 GB | 13 tok/s4-bit quantization |
| Ling 3.0 TinyMoE | 7.9Bactive 1.3B | 4.7 GB | 58 tok/s4-bit quantization |
| Fara 1.5 4BDense | 4B | 2.4 GB | 26 tok/s4-bit quantization |
| Gemma 3 4BDense | 4B | 2.4 GB | 26 tok/s4-bit quantization |
| Qwen 3 4B Instruct 2507Dense | 4B | 2.4 GB | 26 tok/s4-bit quantization |
| Qwen 3.5 4BDense | 4B | 2.4 GB | 26 tok/s4-bit quantization |
| Spark X2.5 4BDense | 4B | 2.4 GB | 26 tok/s4-bit quantization |
| Granite 4.1 3BDense | 3B | 1.8 GB | 35 tok/s4-bit quantization |
| Granite 4.2 3BDense | 3B | 1.8 GB | 35 tok/s4-bit quantization |
| Llama 3.2 3B InstructDense | 3B | 1.8 GB | 35 tok/s4-bit quantization |
| Ministral 3 3BDense | 3B | 1.8 GB | 35 tok/s4-bit quantization |
| Nanbeige 4.2 3BDense | 3B | 1.8 GB | 35 tok/s4-bit quantization |
Tests Qwen3.8-27B GSQ-RCO IQ2_XS (~8.4GB, ~2.50 BPW) versus the 53.8GB BF16 original on an Apple M4 Mac Mini with 24GB RAM.
Explores running a 26B parameter MoE model on limited memory, comparing weight sizes across FP16/BF16, 8-bit, 4-bit, 3-bit, and 2-bit quantization levels.
Tests Bonsai 27B 1-bit quantized versus Qwen3.5 9B on a Mac Mini M4 with 16GB RAM during general chat and within the Pi Coding Agent workflow.
Demonstrates setting up Hermes Agent with Qwen 3.5B via Ollama and LM Studio with Locally on Mac Mini M4, plus comparing local model speed against an M5 Max MacBook Pro.
Tests whether a base 16GB Mac Mini M4 can sufficiently power local LLMs integrated with Pi Coding Agent for actual coding tasks under real-world local AI workloads.