Local generation speed estimates
LLM models
Compare LLMs by approximate model size, Q4 memory needs and the weakest listed hardware expected to reach two local generation thresholds: low-speed use and good-speed use.
Before buying hardware for a model: treat the compatible hardware and generation speed as estimates. Verify the exact model, runtime, quantization, context length, prompt shape and concurrency on the hardware you plan to buy. Check recent hardware reviews, hands-on tests, benchmarks and benchmark videos for similar setups.
Estimation assumptions: Q4 quantization, full-memory fit, no offloading and generation speed only. Use the model details page to compare concrete compatible hardware before deciding what to run or buy.
111 models found
Showing matching model estimates
| LLM models by minimum listed hardware speed | ||||
|---|---|---|---|---|
| Model | Size | Approx. memory 4-bit quantization |
Low-speed fit >= 15 tok/s |
Good-speed fit >= 40 tok/s |
Open-weight / local-fit models
Listed hardware reaches at least the low-speed local generation threshold.
These models have at least one concrete hardware entry estimated to run locally at 15 tok/s or faster under the table assumptions. | ||||
| Granite 4.1 3BDenseModel source: provider | 3B | 1.8 GB | DDR4 16GB Computer15 tok/s | RTX 3060 12GB101 tok/s |
| Granite 4.2 3BDenseModel source: provider | 3B | 1.8 GB | DDR4 16GB Computer15 tok/s | RTX 3060 12GB101 tok/s |
| Llama 3.2 3B InstructDenseModel source: provider | 3B | 1.8 GB | DDR4 16GB Computer15 tok/s | RTX 3060 12GB101 tok/s |
| Ministral 3 3BDenseModel source: provider | 3B | 1.8 GB | DDR4 16GB Computer15 tok/s | RTX 3060 12GB101 tok/s |
| Nanbeige 4.2 3BDenseModel source: provider | 3B | 1.8 GB | DDR4 16GB Computer15 tok/s | RTX 3060 12GB101 tok/s |
| Fara 1.5 4BDenseModel source: provider | 4B | 2.4 GB | DDR5 16GB Computer20 tok/s | RTX 3060 12GB77 tok/s |
| Gemma 3 4BDenseModel source: provider | 4B | 2.4 GB | DDR5 16GB Computer20 tok/s | RTX 3060 12GB77 tok/s |
| Qwen 3 4B Instruct 2507DenseModel source: provider | 4B | 2.4 GB | DDR5 16GB Computer20 tok/s | RTX 3060 12GB77 tok/s |
| Qwen 3.5 4BDenseModel source: provider | 4B | 2.4 GB | DDR5 16GB Computer20 tok/s | RTX 3060 12GB77 tok/s |
| Spark X2.5 4BDenseModel source: provider | 4B | 2.4 GB | DDR5 16GB Computer20 tok/s | RTX 3060 12GB77 tok/s |
| Ling 3.0 TinyMoEModel source: provider | 7.9Bactive 1.3B | 4.7 GB | DDR4 16GB Computer26 tok/s | DDR5 16GB Computer44 tok/s |
| Granite 4.1 8BDenseModel source: provider | 8B | 4.8 GB | RTX 3060 12GB40 tok/s | RTX 3060 12GB40 tok/s |
| Granite 4.2 8BDenseModel source: provider | 8B | 4.8 GB | RTX 3060 12GB40 tok/s | RTX 3060 12GB40 tok/s |
| Qwen 3 8BDenseModel source: provider | 8B | 4.8 GB | RTX 3060 12GB40 tok/s | RTX 3060 12GB40 tok/s |
| Fara 1.5 9BDenseModel source: provider | 9B | 5.4 GB | RTX 3060 12GB35 tok/s | RTX 5060 Ti 16GB44 tok/s |
| Ornith 1.0 9BDenseModel source: derived | 9B | 5.4 GB | RTX 3060 12GB35 tok/s | RTX 5060 Ti 16GB44 tok/s |
| Ornith 1.5 9BDenseModel source: derived | 9B | 5.4 GB | RTX 3060 12GB35 tok/s | RTX 5060 Ti 16GB44 tok/s |
| Qwen 3.5 9BDenseModel source: provider | 9B | 5.4 GB | RTX 3060 12GB35 tok/s | RTX 5060 Ti 16GB44 tok/s |
| Qwythos 9BDenseModel source: derived | 9B | 5.4 GB | RTX 3060 12GB35 tok/s | RTX 5060 Ti 16GB44 tok/s |
| Gemma 3 12BDenseModel source: provider | 12B | 7.2 GB | RTX 3060 12GB26 tok/s | RTX 508069 tok/s |
| Gemma 4 12BDenseModel source: provider | 12B | 7.2 GB | RTX 3060 12GB26 tok/s | RTX 508069 tok/s |
| Qwen 3 14BDenseModel source: provider | 14B | 8.4 GB | RTX 3060 12GB23 tok/s | RTX 508060 tok/s |
| Instella-MoE 16B ThinkMoEModel source: provider | 16Bactive 2.8B | 9.6 GB | DDR5 16GB Computer21 tok/s | RTX 2000 Ada51 tok/s |
| GPT-OSS 20BMoEModel source: provider | 20.9Bactive 3.6B | 12.5 GB | DDR5 16GB Computer16 tok/s | RTX 2000 Ada40 tok/s |
| Gemma 4 26BMoEModel source: provider | 26Bactive 4B | 15.6 GB | Mac mini M420 tok/s | M4 Pro44 tok/s |
| Fara 1.5 27BDenseModel source: provider | 27B | 16.2 GB | RTX 3090 Ti33 tok/s | RTX 509058 tok/s |
| Qwen 3.6 27BDenseModel source: provider | 27B | 16.2 GB | RTX 3090 Ti33 tok/s | RTX 509058 tok/s |
| Qwen 3.8 27BDenseModel source: provider | 27B | 16.2 GB | RTX 3090 Ti33 tok/s | RTX 509058 tok/s |
| GLM 4.7 FlashMoEModel source: provider | 30Bactive 3B | 18 GB | Mac mini M426 tok/s | M4 Pro58 tok/s |
| Nemotron 3.5 LightningMoEModel source: provider | 30Bactive 3B | 18 GB | Mac mini M426 tok/s | M4 Pro58 tok/s |
| Granite 4.1 30BDenseModel source: provider | 30B | 18 GB | RTX 3090 Ti30 tok/s | RTX 509052 tok/s |
| Granite 4.2 30BDenseModel source: provider | 30B | 18 GB | RTX 3090 Ti30 tok/s | RTX 509052 tok/s |
| Muse GlimmerDenseModel source: provider | 30B | 18 GB | RTX 3090 Ti30 tok/s | RTX 509052 tok/s |
| Qwen 3 Coder 30BMoEModel source: provider | 30.5Bactive 3.3B | 18.3 GB | Mac mini M424 tok/s | M4 Pro53 tok/s |
| Gemma 4 31BDenseModel source: provider | 31B | 18.6 GB | RTX 3090 Ti29 tok/s | RTX 509050 tok/s |
| Qwen 2.5 32BDenseModel source: provider | 32.5B | 19.5 GB | RTX 3090 Ti27 tok/s | RTX 509048 tok/s |
| Laguna XS 2.1MoEModel source: provider | 33Bactive 3B | 19.8 GB | Mac mini M426 tok/s | M4 Pro58 tok/s |
| Ornith 1.0 35BMoEModel source: derived | 35Bactive 3B | 21 GB | Mac mini M426 tok/s | M4 Pro58 tok/s |
| Ornith 1.5 35BMoEModel source: derived | 35Bactive 3B | 21 GB | Mac mini M426 tok/s | M4 Pro58 tok/s |
| Qwen 3.6 35BMoEModel source: provider | 35Bactive 3B | 21 GB | Mac mini M426 tok/s | M4 Pro58 tok/s |
| Qwen 3 Coder Next 80BMoEModel source: provider | 80Bactive 3B | 48 GB | M5 Pro64 tok/s | M5 Pro64 tok/s |
| Sarvam 105BMoEModel source: provider | 105Bactive 10.3B | 63 GB | Radeon 8060S 96GB16 tok/s | M3 Ultra51 tok/s |
| GPT-OSS 120BMoEModel source: provider | 116.8Bactive 5.1B | 70.1 GB | Radeon 8060S 96GB33 tok/s | M3 Ultra96 tok/s |
| Laguna S 2.1MoEModel source: provider | 118Bactive 8B | 70.8 GB | Radeon 8060S 96GB21 tok/s | M3 Ultra64 tok/s |
| Mistral Small 4MoEModel source: provider | 119Bactive 6.5B | 71.4 GB | Radeon 8060S 96GB26 tok/s | M3 Ultra77 tok/s |
| Nemotron 3 SuperMoEModel source: provider | 120Bactive 12B | 72 GB | DGX Spark15 tok/s | M3 Ultra44 tok/s |
| Qwen 3.5 122BMoEModel source: provider | 122Bactive 10B | 73.2 GB | Radeon 8060S 96GB17 tok/s | M3 Ultra52 tok/s |
| Ling 3.0 Flash VLMoEModel source: provider | 124Bactive 5.5B | 74.4 GB | Radeon 8060S 96GB31 tok/s | M3 Ultra90 tok/s |
| Ling 3.0 FlashMoEModel source: provider | 124Bactive 5.1B | 74.4 GB | Radeon 8060S 96GB33 tok/s | M3 Ultra96 tok/s |
| Qwen 3.8 Flash NextMoEModel source: provider | 125Bactive 6B | 75 GB | Radeon 8060S 96GB28 tok/s | M3 Ultra83 tok/s |
| Mistral Medium 3.5DenseModel source: provider | 128B | 76.8 GB | DGX Station48 tok/s | DGX Station48 tok/s |
| Step 3.7 FlashMoEModel source: provider | 198Bactive 11B | 118.8 GB | DGX Spark16 tok/s | M5 Ultra68 tok/s |
| Command A+MoEModel source: provider | 218Bactive 25B | 130.8 GB | M5 Ultra31 tok/s | DGX Station153 tok/s |
| Laguna M.1MoEModel source: provider | 225Bactive 23B | 135 GB | M5 Ultra34 tok/s | DGX Station163 tok/s |
| Qwen 3 235BMoEModel source: provider | 235Bactive 22B | 141 GB | M5 Ultra36 tok/s | DGX Station168 tok/s |
| Solar Open 2MoEModel source: provider | 250Bactive 15B | 150 GB | M5 Ultra51 tok/s | M5 Ultra51 tok/s |
| Inkling SmallMoEModel source: provider | 276Bactive 12B | 165.6 GB | M5 Ultra63 tok/s | M5 Ultra63 tok/s |
| DeepSeek V4 FlashMoEModel source: provider | 284Bactive 13B | 170.4 GB | M5 Ultra58 tok/s | M5 Ultra58 tok/s |
| Hy3MoEModel source: provider | 295Bactive 21B | 177 GB | M5 Ultra37 tok/s | DGX Station174 tok/s |
| MiMo V2.5MoEModel source: provider | 310Bactive 15B | 186 GB | M5 Ultra51 tok/s | M5 Ultra51 tok/s |
| GLM 5.3 FlashMoEModel source: provider | 320Bactive 18B | 192 GB | M5 Ultra43 tok/s | M5 Ultra43 tok/s |
| Ornith 1.5 397BMoEModel source: derived | 397Bactive 17B | 238.2 GB | M5 Ultra46 tok/s | M5 Ultra46 tok/s |
| Qwen 3.5 397BMoEModel source: provider | 397Bactive 17B | 238.2 GB | M5 Ultra46 tok/s | M5 Ultra46 tok/s |
| MiniMax M3MoEModel source: provider | 428Bactive 23B | 256.8 GB | DGX H200119 tok/s | DGX Station163 tok/s |
| Qwen 3 Coder 480BMoEModel source: provider | 480Bactive 35B | 288 GB | DGX Station117 tok/s | DGX Station117 tok/s |
| Nemotron 3 UltraMoEModel source: provider | 550Bactive 55B | 330 GB | DGX Station79 tok/s | DGX Station79 tok/s |
| DeepSeek V4.1 FlashMoEModel source: provider | 552Bactive 16B | 331.2 GB | DGX H200159 tok/s | DGX Station211 tok/s |
| GLM 5.1MoEModel source: provider | 744Bactive 40B | 446.4 GB | DGX H20074 tok/s | DGX Station104 tok/s |
| GLM 5.2MoEModel source: provider | 744Bactive 40B | 446.4 GB | DGX H20074 tok/s | DGX Station104 tok/s |
| Hy4MoEModel source: provider | 770Bactive 49B | 462 GB | DGX H20062 tok/s | DGX Station88 tok/s |
| Kimi K2.6MoEModel source: provider | 1000Bactive 32B | 600 GB | DGX H20090 tok/s | DGX Station125 tok/s |
| Kimi K2.7MoEModel source: provider | 1000Bactive 32B | 600 GB | DGX H20090 tok/s | DGX Station125 tok/s |
| MiMo V2.5 ProMoEModel source: provider | 1020Bactive 42B | 612 GB | DGX H20071 tok/s | DGX Station100 tok/s |
| DeepSeek V4 ProMoEModel source: provider | 1600Bactive 49B | 960 GB | DGX H20062 tok/s | DGX H20062 tok/s |
| Qwen 3.8 2.4TMoEModel source: provider | 2400Bactive 95B | 1440 GB | GB200 NVL7254 tok/s | GB200 NVL7254 tok/s |
| Kimi K3MoEModel source: provider | 2800Bactive 104B | 1680 GB | GB200 NVL7249 tok/s | GB200 NVL7249 tok/s |
| Model | Size | Approx. memory 4-bit quantization |
Low-speed fit >= 15 tok/s |
Good-speed fit >= 40 tok/s |
API / no listed local fit
No listed hardware reaches the local generation thresholds.
This section includes hosted/proprietary-style models, models without enough public sizing data, and models that exceed the currently listed local hardware examples. | ||||
| ERNIE 5.1Model source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |
| Fable 5Model source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |
| Fable 5.1Model source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |
| Gemini 3.5 FlashModel source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |
| Gemini 3.5 Flash-LiteModel source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |
| Gemini 3.6 FlashModel source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |
| Gemini 3.7 FlashModel source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |
| Gemini 3.8 FlashModel source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |
| GLM 5.3Model source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |
| GPT 5.4Model source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |
| GPT 5.4 MiniModel source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |
| GPT 5.4 NanoModel source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |
| GPT 5.4 ProModel source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |
| GPT 5.5Model source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |
| GPT 5.5 ProModel source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |
| GPT 5.6 LunaModel source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |
| GPT 5.6 SolModel source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |
| GPT 5.6 TerraModel source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |
| GPT 6 AstraModel source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |
| Grok 4.20Model source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |
| Grok 4.5Model source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |
| Grok 4.6Model source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |
| MAI Thinking 1Model source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |
| Muse Spark 1.1Model source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |
| Muse Spark 1.2Model source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |
| Muse Spark 1.3Model source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |
| Mythos 5Model source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |
| Opus 4.7Model source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |
| Opus 4.8Model source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |
| Opus 5Model source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |
| Qwen 3.7 MaxModel source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |
| Qwen 3.8 MaxModel source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |
| Qwen 3.8 Omni FlashModel source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |
| Solar Pro 4Model source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |
| Sonnet 5Model source: provider | not available | not available | Hosted / unknown local fit | Hosted / unknown local fit |