| Hardware profile | ||||
|---|---|---|---|---|
| Hardware | Memory | Bandwidth | Computing power | vLLM |
| Raspberry Pi 5Raspberry Pi 5 16 GB | 16 GBLPDDR4X RAM | 17 GB/s | not available | no |
| Compatible models Estimated speeds, not benchmark results: calculated from memory bandwidth and model size. Real results can differ significantly because there is no precise formula for deriving LLM generation speed from hardware specifications alone. | |||
|---|---|---|---|
| Model | Size | Approx. Q4 memory | Estimated generation |
| GPT-OSS 20BMoE | 20.9Bactive 3.6B | 12.5 GB | 3 tok/s4-bit quantization |
| Instella-MoE 16B ThinkMoE | 16Bactive 2.8B | 9.6 GB | 4 tok/s4-bit quantization |
| Qwen 3 14BDense | 14B | 8.4 GB | 1 tok/s4-bit quantization |
| Gemma 3 12BDense | 12B | 7.2 GB | 1 tok/s4-bit quantization |
| Gemma 4 12BDense | 12B | 7.2 GB | 1 tok/s4-bit quantization |
| Fara 1.5 9BDense | 9B | 5.4 GB | 1 tok/s4-bit quantization |
| Ornith 1.0 9BDense | 9B | 5.4 GB | 1 tok/s4-bit quantization |
| Ornith 1.5 9BDense | 9B | 5.4 GB | 1 tok/s4-bit quantization |
| Qwen 3.5 9BDense | 9B | 5.4 GB | 1 tok/s4-bit quantization |
| Qwythos 9BDense | 9B | 5.4 GB | 1 tok/s4-bit quantization |
| Granite 4.1 8BDense | 8B | 4.8 GB | 1 tok/s4-bit quantization |
| Granite 4.2 8BDense | 8B | 4.8 GB | 1 tok/s4-bit quantization |
| Qwen 3 8BDense | 8B | 4.8 GB | 1 tok/s4-bit quantization |
| LFM 2.5 8BMoE | 8Bactive 1B | 4.8 GB | 11 tok/s4-bit quantization |
| Ling 3.0 TinyMoE | 7.9Bactive 1.3B | 4.7 GB | 9 tok/s4-bit quantization |
| Fara 1.5 4BDense | 4B | 2.4 GB | 3 tok/s4-bit quantization |
| Gemma 3 4BDense | 4B | 2.4 GB | 3 tok/s4-bit quantization |
| Qwen 3 4B Instruct 2507Dense | 4B | 2.4 GB | 3 tok/s4-bit quantization |
| Qwen 3.5 4BDense | 4B | 2.4 GB | 3 tok/s4-bit quantization |
| Spark X2.5 4BDense | 4B | 2.4 GB | 3 tok/s4-bit quantization |
| Granite 4.1 3BDense | 3B | 1.8 GB | 5 tok/s4-bit quantization |
| Granite 4.2 3BDense | 3B | 1.8 GB | 5 tok/s4-bit quantization |
| Llama 3.2 3B InstructDense | 3B | 1.8 GB | 5 tok/s4-bit quantization |
| Ministral 3 3BDense | 3B | 1.8 GB | 5 tok/s4-bit quantization |
| Nanbeige 4.2 3BDense | 3B | 1.8 GB | 5 tok/s4-bit quantization |
| LFM 3 1.7BDense | 1.7B | 1 GB | 9 tok/s4-bit quantization |
Eight local AI models were tested on a Raspberry Pi 5 using llama.cpp across summarization, sorting, email parsing, and tool calling, measuring tokens per second and time to first word.