| Hardware profile | ||||
|---|---|---|---|---|
| Hardware | Memory | Bandwidth | Computing power | vLLM |
| RTX 3060 12GBNVIDIA GeForce RTX 3060 12 GB desktop GPU | 12 GBGPU VRAM | 360 GB/s | ~101 TOPS | yes |
| Compatible models Estimated speeds, not benchmark results: calculated from memory bandwidth and model size. Real results can differ significantly because there is no precise formula for deriving LLM generation speed from hardware specifications alone. | |||
|---|---|---|---|
| Model | Size | Approx. Q4 memory | Estimated generation |
| Instella-MoE 16B ThinkMoE | 16Bactive 2.8B | 9.6 GB | 79 tok/s4-bit quantization |
| Qwen 3 14BDense | 14B | 8.4 GB | 23 tok/s4-bit quantization |
| Gemma 3 12BDense | 12B | 7.2 GB | 26 tok/s4-bit quantization |
| Gemma 4 12BDense | 12B | 7.2 GB | 26 tok/s4-bit quantization |
| Fara 1.5 9BDense | 9B | 5.4 GB | 35 tok/s4-bit quantization |
| Ornith 1.0 9BDense | 9B | 5.4 GB | 35 tok/s4-bit quantization |
| Ornith 1.5 9BDense | 9B | 5.4 GB | 35 tok/s4-bit quantization |
| Qwen 3.5 9BDense | 9B | 5.4 GB | 35 tok/s4-bit quantization |
| Qwythos 9BDense | 9B | 5.4 GB | 35 tok/s4-bit quantization |
| Granite 4.1 8BDense | 8B | 4.8 GB | 40 tok/s4-bit quantization |
| Granite 4.2 8BDense | 8B | 4.8 GB | 40 tok/s4-bit quantization |
| Qwen 3 8BDense | 8B | 4.8 GB | 40 tok/s4-bit quantization |
| Ling 3.0 TinyMoE | 7.9Bactive 1.3B | 4.7 GB | 150 tok/s4-bit quantization |
| Fara 1.5 4BDense | 4B | 2.4 GB | 77 tok/s4-bit quantization |
| Gemma 3 4BDense | 4B | 2.4 GB | 77 tok/s4-bit quantization |
| Qwen 3 4B Instruct 2507Dense | 4B | 2.4 GB | 77 tok/s4-bit quantization |
| Qwen 3.5 4BDense | 4B | 2.4 GB | 77 tok/s4-bit quantization |
| Spark X2.5 4BDense | 4B | 2.4 GB | 77 tok/s4-bit quantization |
| Granite 4.1 3BDense | 3B | 1.8 GB | 101 tok/s4-bit quantization |
| Granite 4.2 3BDense | 3B | 1.8 GB | 101 tok/s4-bit quantization |
| Llama 3.2 3B InstructDense | 3B | 1.8 GB | 101 tok/s4-bit quantization |
| Ministral 3 3BDense | 3B | 1.8 GB | 101 tok/s4-bit quantization |
| Nanbeige 4.2 3BDense | 3B | 1.8 GB | 101 tok/s4-bit quantization |
Demonstrates optimizing a local voice agent in Pithagoras using Qwen3.6-35B-A3B, Breeze-TTS-2, Whisper, and Silero VAD on an RTX 3060 via overlapping pipelines and quantization.
Benchmarks unsloth/Qwen3.8-Flash-Next-GGUF on RTX 3060 using llama.cpp and thecodacus/spec-wins coding lab with contradictory specs and decoys.
Demonstrates Qwen3.6-35B-A3B inference on RTX 3060 using llama.cpp MoE expert caching combined with speculative decoding, achieving 70-80 tok/s via --moe-cache-profile flags.
Evaluates Qwen3.8-27B inference via llama.cpp RPC across RTX 3060 and Apple Silicon Mac, comparing throughput metrics for single-card versus distributed layer loading.
Compares Laguna S 2.1 and Qwen 3.6 35B-A3B on an RTX 3060 using llama.cpp, evaluating debugging accuracy and animation generation tasks via manual testing.
Tests Bonsai 27B binary and ternary variants against Qwen 3.6 35B-A3B and Gemma 4 12B on an RTX 3060 using llama.cpp, measuring generation speed, total memory footprint, and performance on web design
Claude Fable 5 optimized llama.cpp for Qwen3.6 35B-A3B on an RTX 3060, achieving a 64.5% prefill speedup via four patches, with results verified through self-benchmarking.
Runs DiffusionGemma via llama.cpp on RTX 3060, achieving ~44 tok/s with MoE offloading and quantization. Covers diffusion text mechanics, strengths in code/OCR/Sudoku, and local setup steps.