| Hardware profile | ||||
|---|---|---|---|---|
| Hardware | Memory | Bandwidth | Computing power | vLLM |
| RTX 2000 AdaNVIDIA RTX 2000 Ada Generation 16 GB | 16 GBGPU VRAM | 224 GB/s | ~191.9 TOPS | yes |
| Compatible models Estimated speeds, not benchmark results: calculated from memory bandwidth and model size. Real results can differ significantly because there is no precise formula for deriving LLM generation speed from hardware specifications alone. | |||
|---|---|---|---|
| Model | Size | Approx. Q4 memory | Estimated generation |
| GPT-OSS 20BMoE | 20.9Bactive 3.6B | 12.5 GB | 40 tok/s4-bit quantization |
| Instella-MoE 16B ThinkMoE | 16Bactive 2.8B | 9.6 GB | 51 tok/s4-bit quantization |
| Qwen 3 14BDense | 14B | 8.4 GB | 14 tok/s4-bit quantization |
| Gemma 3 12BDense | 12B | 7.2 GB | 16 tok/s4-bit quantization |
| Gemma 4 12BDense | 12B | 7.2 GB | 16 tok/s4-bit quantization |
| Fara 1.5 9BDense | 9B | 5.4 GB | 22 tok/s4-bit quantization |
| Ornith 1.0 9BDense | 9B | 5.4 GB | 22 tok/s4-bit quantization |
| Ornith 1.5 9BDense | 9B | 5.4 GB | 22 tok/s4-bit quantization |
| Qwen 3.5 9BDense | 9B | 5.4 GB | 22 tok/s4-bit quantization |
| Qwythos 9BDense | 9B | 5.4 GB | 22 tok/s4-bit quantization |
| Granite 4.1 8BDense | 8B | 4.8 GB | 25 tok/s4-bit quantization |
| Granite 4.2 8BDense | 8B | 4.8 GB | 25 tok/s4-bit quantization |
| Qwen 3 8BDense | 8B | 4.8 GB | 25 tok/s4-bit quantization |
| LFM 2.5 8BMoE | 8Bactive 1B | 4.8 GB | 126 tok/s4-bit quantization |
| Ling 3.0 TinyMoE | 7.9Bactive 1.3B | 4.7 GB | 102 tok/s4-bit quantization |
| Fara 1.5 4BDense | 4B | 2.4 GB | 49 tok/s4-bit quantization |
| Gemma 3 4BDense | 4B | 2.4 GB | 49 tok/s4-bit quantization |
| Qwen 3 4B Instruct 2507Dense | 4B | 2.4 GB | 49 tok/s4-bit quantization |
| Qwen 3.5 4BDense | 4B | 2.4 GB | 49 tok/s4-bit quantization |
| Spark X2.5 4BDense | 4B | 2.4 GB | 49 tok/s4-bit quantization |
| Granite 4.1 3BDense | 3B | 1.8 GB | 65 tok/s4-bit quantization |
| Granite 4.2 3BDense | 3B | 1.8 GB | 65 tok/s4-bit quantization |
| Llama 3.2 3B InstructDense | 3B | 1.8 GB | 65 tok/s4-bit quantization |
| Ministral 3 3BDense | 3B | 1.8 GB | 65 tok/s4-bit quantization |
| Nanbeige 4.2 3BDense | 3B | 1.8 GB | 65 tok/s4-bit quantization |
| LFM 3 1.7BDense | 1.7B | 1 GB | 110 tok/s4-bit quantization |
Raon 1 Mini is evaluated via Performance, Memory, Reasoning, HumanEval Remix, Kanban, Sand Physics, Dungeon Crawler, Blender, and Godot tests using RTX 2000 and RTX 5090 hardware.
Tests InternScience/Sage-A1.5-Q4_K_M-GGUF on RTX 2000 and RTX 5090 using Performance, Memory, Reasoning, HumanEval Remix, Seat Planner, Sand Physics, Dungeon Crawler, Blender, and Godot benchmarks.
Tests Flint by RecursiveSI using Performance, Memory, Reasoning, HumanEval Remix, Kanban, Sand Physics, Dungeon Crawler, Blender and Godot on RTX 2000 and RTX 5090.
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder is evaluated via performance on RTX 2000 16GB, memory, reasoning, HumanEval Remix, Kanban, Sand Physics, Dungeon Crawler, Blender and Godot.
Tests Qwen 3.8 27B Ridge from Empero AI on 16GB GPUs using benchmarks including OpenAI Human Eval, Kanban, Sand Physics, Dungeon Crawler, Blender and Godot.
Tests Ornith-1.5-35B-A3B-GGUF performance, memory, agency, OpenAI Human Eval, Kanban, Sand Physics, Dungeon Crawler, Blender and Godot.