| Hardware profile | ||||
|---|---|---|---|---|
| Hardware | Memory | Bandwidth | Computing power | vLLM |
| RTX 4060 Ti 16GBLow-cost NVIDIA 16 GB VRAM desktop GPU baseline | 16 GBGPU VRAM | 288 GB/s | ~353 TOPS | yes |
| Compatible models Estimated speeds, not benchmark results: calculated from memory bandwidth and model size. Real results can differ significantly because there is no precise formula for deriving LLM generation speed from hardware specifications alone. | |||
|---|---|---|---|
| Model | Size | Approx. Q4 memory | Estimated generation |
| GPT-OSS 20BMoE | 20.9Bactive 3.6B | 12.5 GB | 51 tok/s4-bit quantization |
| Instella-MoE 16B ThinkMoE | 16Bactive 2.8B | 9.6 GB | 64 tok/s4-bit quantization |
| Qwen 3 14BDense | 14B | 8.4 GB | 18 tok/s4-bit quantization |
| Gemma 3 12BDense | 12B | 7.2 GB | 21 tok/s4-bit quantization |
| Gemma 4 12BDense | 12B | 7.2 GB | 21 tok/s4-bit quantization |
| Fara 1.5 9BDense | 9B | 5.4 GB | 28 tok/s4-bit quantization |
| Ornith 1.0 9BDense | 9B | 5.4 GB | 28 tok/s4-bit quantization |
| Ornith 1.5 9BDense | 9B | 5.4 GB | 28 tok/s4-bit quantization |
| Qwen 3.5 9BDense | 9B | 5.4 GB | 28 tok/s4-bit quantization |
| Qwythos 9BDense | 9B | 5.4 GB | 28 tok/s4-bit quantization |
| Granite 4.1 8BDense | 8B | 4.8 GB | 32 tok/s4-bit quantization |
| Granite 4.2 8BDense | 8B | 4.8 GB | 32 tok/s4-bit quantization |
| Qwen 3 8BDense | 8B | 4.8 GB | 32 tok/s4-bit quantization |
| Ling 3.0 TinyMoE | 7.9Bactive 1.3B | 4.7 GB | 125 tok/s4-bit quantization |
| Fara 1.5 4BDense | 4B | 2.4 GB | 62 tok/s4-bit quantization |
| Gemma 3 4BDense | 4B | 2.4 GB | 62 tok/s4-bit quantization |
| Qwen 3 4B Instruct 2507Dense | 4B | 2.4 GB | 62 tok/s4-bit quantization |
| Qwen 3.5 4BDense | 4B | 2.4 GB | 62 tok/s4-bit quantization |
| Spark X2.5 4BDense | 4B | 2.4 GB | 62 tok/s4-bit quantization |
| Granite 4.1 3BDense | 3B | 1.8 GB | 82 tok/s4-bit quantization |
| Granite 4.2 3BDense | 3B | 1.8 GB | 82 tok/s4-bit quantization |
| Llama 3.2 3B InstructDense | 3B | 1.8 GB | 82 tok/s4-bit quantization |
| Ministral 3 3BDense | 3B | 1.8 GB | 82 tok/s4-bit quantization |
| Nanbeige 4.2 3BDense | 3B | 1.8 GB | 82 tok/s4-bit quantization |
Evaluates entirely-in-VRAM configurations for 16GB GPUs using Base Qwen and Qwen 3.8 27B GSQ-RCO against unsloth quants, demonstrating live iOS coding performance.