| Hardware profile | ||||
|---|---|---|---|---|
| Hardware | Memory | Bandwidth | Computing power | vLLM |
| RTX 2000 AdaNVIDIA RTX 2000 Ada Generation 16 GB | 16 GBGPU VRAM | 224 GB/s | ~191.9 TOPS | yes |
| Compatible models Estimated speeds, not benchmark results: calculated from memory bandwidth and model size. Real results can differ significantly because there is no precise formula for deriving LLM generation speed from hardware specifications alone. | |||
|---|---|---|---|
| Model | Size | Approx. Q4 memory | Estimated generation |
| GPT-OSS 20BMoE | 20.9Bactive 3.6B | 12.5 GB | 40 tok/s4-bit quantization |
| Instella-MoE 16B ThinkMoE | 16Bactive 2.8B | 9.6 GB | 51 tok/s4-bit quantization |
| Qwen 3 14BDense | 14B | 8.4 GB | 14 tok/s4-bit quantization |
| Gemma 3 12BDense | 12B | 7.2 GB | 16 tok/s4-bit quantization |
| Gemma 4 12BDense | 12B | 7.2 GB | 16 tok/s4-bit quantization |
| Fara 1.5 9BDense | 9B | 5.4 GB | 22 tok/s4-bit quantization |
| Ornith 1.0 9BDense | 9B | 5.4 GB | 22 tok/s4-bit quantization |
| Ornith 1.5 9BDense | 9B | 5.4 GB | 22 tok/s4-bit quantization |
| Qwen 3.5 9BDense | 9B | 5.4 GB | 22 tok/s4-bit quantization |
| Qwythos 9BDense | 9B | 5.4 GB | 22 tok/s4-bit quantization |
| Granite 4.1 8BDense | 8B | 4.8 GB | 25 tok/s4-bit quantization |
| Granite 4.2 8BDense | 8B | 4.8 GB | 25 tok/s4-bit quantization |
| Qwen 3 8BDense | 8B | 4.8 GB | 25 tok/s4-bit quantization |
| Ling 3.0 TinyMoE | 7.9Bactive 1.3B | 4.7 GB | 102 tok/s4-bit quantization |
| Fara 1.5 4BDense | 4B | 2.4 GB | 49 tok/s4-bit quantization |
| Gemma 3 4BDense | 4B | 2.4 GB | 49 tok/s4-bit quantization |
| Qwen 3 4B Instruct 2507Dense | 4B | 2.4 GB | 49 tok/s4-bit quantization |
| Qwen 3.5 4BDense | 4B | 2.4 GB | 49 tok/s4-bit quantization |
| Spark X2.5 4BDense | 4B | 2.4 GB | 49 tok/s4-bit quantization |
| Granite 4.1 3BDense | 3B | 1.8 GB | 65 tok/s4-bit quantization |
| Granite 4.2 3BDense | 3B | 1.8 GB | 65 tok/s4-bit quantization |
| Llama 3.2 3B InstructDense | 3B | 1.8 GB | 65 tok/s4-bit quantization |
| Ministral 3 3BDense | 3B | 1.8 GB | 65 tok/s4-bit quantization |
| Nanbeige 4.2 3BDense | 3B | 1.8 GB | 65 tok/s4-bit quantization |
Tests Qwen 3.8 27B Ridge from Empero AI on 16GB GPUs using benchmarks including OpenAI Human Eval, Kanban, Sand Physics, Dungeon Crawler, Blender and Godot.
Tests Ornith-1.5-35B-A3B-GGUF performance, memory, agency, OpenAI Human Eval, Kanban, Sand Physics, Dungeon Crawler, Blender and Godot.