| Hardware profile | ||||
|---|---|---|---|---|
| Hardware | Memory | Bandwidth | Computing power | vLLM |
| RTX 3060 12GBNVIDIA GeForce RTX 3060 12 GB desktop GPU | 12 GBGPU VRAM | 360 GB/s | ~101 TOPS | yes |
| Compatible models Estimated speeds, not benchmark results: calculated from memory bandwidth and model size. Real results can differ significantly because there is no precise formula for deriving LLM generation speed from hardware specifications alone. | |||
|---|---|---|---|
| Model | Size | Approx. Q4 memory | Estimated generation |
| Instella-MoE 16B ThinkMoE | 16Bactive 2.8B | 9.6 GB | 79 tok/s4-bit quantization |
| Qwen 3 14BDense | 14B | 8.4 GB | 23 tok/s4-bit quantization |
| Gemma 3 12BDense | 12B | 7.2 GB | 26 tok/s4-bit quantization |
| Gemma 4 12BDense | 12B | 7.2 GB | 26 tok/s4-bit quantization |
| Fara 1.5 9BDense | 9B | 5.4 GB | 35 tok/s4-bit quantization |
| Ornith 1.0 9BDense | 9B | 5.4 GB | 35 tok/s4-bit quantization |
| Ornith 1.5 9BDense | 9B | 5.4 GB | 35 tok/s4-bit quantization |
| Qwen 3.5 9BDense | 9B | 5.4 GB | 35 tok/s4-bit quantization |
| Qwythos 9BDense | 9B | 5.4 GB | 35 tok/s4-bit quantization |
| Granite 4.1 8BDense | 8B | 4.8 GB | 40 tok/s4-bit quantization |
| Granite 4.2 8BDense | 8B | 4.8 GB | 40 tok/s4-bit quantization |
| Qwen 3 8BDense | 8B | 4.8 GB | 40 tok/s4-bit quantization |
| LFM 2.5 8BMoE | 8Bactive 1B | 4.8 GB | 182 tok/s4-bit quantization |
| Ling 3.0 TinyMoE | 7.9Bactive 1.3B | 4.7 GB | 150 tok/s4-bit quantization |
| Fara 1.5 4BDense | 4B | 2.4 GB | 77 tok/s4-bit quantization |
| Gemma 3 4BDense | 4B | 2.4 GB | 77 tok/s4-bit quantization |
| Qwen 3 4B Instruct 2507Dense | 4B | 2.4 GB | 77 tok/s4-bit quantization |
| Qwen 3.5 4BDense | 4B | 2.4 GB | 77 tok/s4-bit quantization |
| Spark X2.5 4BDense | 4B | 2.4 GB | 77 tok/s4-bit quantization |
| Granite 4.1 3BDense | 3B | 1.8 GB | 101 tok/s4-bit quantization |
| Granite 4.2 3BDense | 3B | 1.8 GB | 101 tok/s4-bit quantization |
| Llama 3.2 3B InstructDense | 3B | 1.8 GB | 101 tok/s4-bit quantization |
| Ministral 3 3BDense | 3B | 1.8 GB | 101 tok/s4-bit quantization |
| Nanbeige 4.2 3BDense | 3B | 1.8 GB | 101 tok/s4-bit quantization |
| LFM 3 1.7BDense | 1.7B | 1 GB | 169 tok/s4-bit quantization |
Tests looping Qwen3-4B, Qwen3.6-35B-A3B and NVIDIA's Dory on RTX 3060 using MMLU-Pro, CommonsenseQA and SWE-bench Verified benchmarks via llama.cpp.
Demonstrates full-duplex voice AI using Qwen3.6-35B-A3B on RTX 3060 via 0.6s micro-turns and LoRA fine-tuning, based on DuplexCascade.
Demonstrates Strata running Qwen 3.8 Flash on RTX 3060, achieving 900+ tok/s prompt processing via a voice agent placing a grocery order in under two minutes.
Tests dual RTX 3060 performance using llama.cpp tensor split and Raylight for Qwen 3.6 35B MoE, Qwen 3.8 27B, and MiniMax H3 video generation within a Proxmox/ZFS environment.
Demonstrates optimizing a local voice agent in Pithagoras using Qwen3.6-35B-A3B, Breeze-TTS-2, Whisper, and Silero VAD on an RTX 3060 via overlapping pipelines and quantization.
Benchmarks unsloth/Qwen3.8-Flash-Next-GGUF on RTX 3060 using llama.cpp and thecodacus/spec-wins coding lab with contradictory specs and decoys.
Demonstrates Qwen3.6-35B-A3B inference on RTX 3060 using llama.cpp MoE expert caching combined with speculative decoding, achieving 70-80 tok/s via --moe-cache-profile flags.
Evaluates Qwen3.8-27B inference via llama.cpp RPC across RTX 3060 and Apple Silicon Mac, comparing throughput metrics for single-card versus distributed layer loading.
Compares Laguna S 2.1 and Qwen 3.6 35B-A3B on an RTX 3060 using llama.cpp, evaluating debugging accuracy and animation generation tasks via manual testing.
Tests Bonsai 27B binary and ternary variants against Qwen 3.6 35B-A3B and Gemma 4 12B on an RTX 3060 using llama.cpp, measuring generation speed, total memory footprint, and performance on web design
Claude Fable 5 optimized llama.cpp for Qwen3.6 35B-A3B on an RTX 3060, achieving a 64.5% prefill speedup via four patches, with results verified through self-benchmarking.
Runs DiffusionGemma via llama.cpp on RTX 3060, achieving ~44 tok/s with MoE offloading and quantization. Covers diffusion text mechanics, strengths in code/OCR/Sudoku, and local setup steps.