| Hardware profile | ||||
|---|---|---|---|---|
| Hardware | Memory | Bandwidth | Computing power | vLLM |
| RTX 3090 TiNVIDIA GeForce RTX 3090 Ti 24 GB | 24 GBGPU VRAM | 1008 GB/s | ~320 TOPS | yes |
| Compatible models Estimated speeds, not benchmark results: calculated from memory bandwidth and model size. Real results can differ significantly because there is no precise formula for deriving LLM generation speed from hardware specifications alone. | |||
|---|---|---|---|
| Model | Size | Approx. Q4 memory | Estimated generation |
| Ornith 1.0 35BMoE | 35Bactive 3B | 21 GB | 173 tok/s4-bit quantization |
| Ornith 1.5 35BMoE | 35Bactive 3B | 21 GB | 173 tok/s4-bit quantization |
| Qwen 3.6 35BMoE | 35Bactive 3B | 21 GB | 173 tok/s4-bit quantization |
| Laguna XS 2.1MoE | 33Bactive 3B | 19.8 GB | 173 tok/s4-bit quantization |
| Qwen 2.5 32BDense | 32.5B | 19.5 GB | 27 tok/s4-bit quantization |
| Gemma 4 31BDense | 31B | 18.6 GB | 29 tok/s4-bit quantization |
| Qwen 3 Coder 30BMoE | 30.5Bactive 3.3B | 18.3 GB | 161 tok/s4-bit quantization |
| Granite 4.1 30BDense | 30B | 18 GB | 30 tok/s4-bit quantization |
| Granite 4.2 30BDense | 30B | 18 GB | 30 tok/s4-bit quantization |
| Muse GlimmerDense | 30B | 18 GB | 30 tok/s4-bit quantization |
| GLM 4.7 FlashMoE | 30Bactive 3B | 18 GB | 173 tok/s4-bit quantization |
| Nemotron 3.5 LightningMoE | 30Bactive 3B | 18 GB | 173 tok/s4-bit quantization |
| Fara 1.5 27BDense | 27B | 16.2 GB | 33 tok/s4-bit quantization |
| Qwen 3.6 27BDense | 27B | 16.2 GB | 33 tok/s4-bit quantization |
| Qwen 3.8 27BDense | 27B | 16.2 GB | 33 tok/s4-bit quantization |
| Gemma 4 26BMoE | 26Bactive 4B | 15.6 GB | 139 tok/s4-bit quantization |
| GPT-OSS 20BMoE | 20.9Bactive 3.6B | 12.5 GB | 151 tok/s4-bit quantization |
| Instella-MoE 16B ThinkMoE | 16Bactive 2.8B | 9.6 GB | 182 tok/s4-bit quantization |
| Qwen 3 14BDense | 14B | 8.4 GB | 62 tok/s4-bit quantization |
| Gemma 3 12BDense | 12B | 7.2 GB | 72 tok/s4-bit quantization |
| Gemma 4 12BDense | 12B | 7.2 GB | 72 tok/s4-bit quantization |
| Fara 1.5 9BDense | 9B | 5.4 GB | 95 tok/s4-bit quantization |
| Ornith 1.0 9BDense | 9B | 5.4 GB | 95 tok/s4-bit quantization |
| Ornith 1.5 9BDense | 9B | 5.4 GB | 95 tok/s4-bit quantization |
| Qwen 3.5 9BDense | 9B | 5.4 GB | 95 tok/s4-bit quantization |
| Qwythos 9BDense | 9B | 5.4 GB | 95 tok/s4-bit quantization |
| Granite 4.1 8BDense | 8B | 4.8 GB | 106 tok/s4-bit quantization |
| Granite 4.2 8BDense | 8B | 4.8 GB | 106 tok/s4-bit quantization |
| Qwen 3 8BDense | 8B | 4.8 GB | 106 tok/s4-bit quantization |
| Ling 3.0 TinyMoE | 7.9Bactive 1.3B | 4.7 GB | 299 tok/s4-bit quantization |
| Fara 1.5 4BDense | 4B | 2.4 GB | 196 tok/s4-bit quantization |
| Gemma 3 4BDense | 4B | 2.4 GB | 196 tok/s4-bit quantization |
| Qwen 3 4B Instruct 2507Dense | 4B | 2.4 GB | 196 tok/s4-bit quantization |
| Qwen 3.5 4BDense | 4B | 2.4 GB | 196 tok/s4-bit quantization |
| Spark X2.5 4BDense | 4B | 2.4 GB | 196 tok/s4-bit quantization |
| Granite 4.1 3BDense | 3B | 1.8 GB | 250 tok/s4-bit quantization |
| Granite 4.2 3BDense | 3B | 1.8 GB | 250 tok/s4-bit quantization |
| Llama 3.2 3B InstructDense | 3B | 1.8 GB | 250 tok/s4-bit quantization |
| Ministral 3 3BDense | 3B | 1.8 GB | 250 tok/s4-bit quantization |
| Nanbeige 4.2 3BDense | 3B | 1.8 GB | 250 tok/s4-bit quantization |
Programming test of Meta's Muse Glimmer 30B model on RTX 3090 Ti using Q4_K_XL and Q4 KV cache quantization, reporting 60 tok/s decode speed and 75% MTP accept rate.
Compares Ornith 35B and Qwen 3.6 35B-A3B using llama.cpp, llama-swap, and OpenCode on dual RTX 3090s while building a street-racing car OS, race-control interface, and live race simulator UI.