Local AI hardware: Radeon 8060S 96GB

Hardware profile
Hardware	Memory	Bandwidth	Computing power	vLLM
Radeon 8060S 96GBAMD Radeon 8060S Graphics with 96 GB allocated unified memory	96 GBUnified RAM	256 GB/s	not available	yes

Compatible models Estimated speeds, not benchmark results: calculated from memory bandwidth and model size. Real results can differ significantly because there is no precise formula for deriving LLM generation speed from hardware specifications alone.
Model	Size	Approx. Q4 memory	Estimated generation
Mistral Medium 3.5Dense	128B	76.8 GB	1 tok/s4-bit quantization
Qwen 3.8 Flash NextMoE	125Bactive 6B	75 GB	28 tok/s4-bit quantization
Ling 3.0 Flash VLMoE	124Bactive 5.5B	74.4 GB	31 tok/s4-bit quantization
Ling 3.0 FlashMoE	124Bactive 5.1B	74.4 GB	33 tok/s4-bit quantization
Qwen 3.5 122BMoE	122Bactive 10B	73.2 GB	17 tok/s4-bit quantization
Nemotron 3 SuperMoE	120Bactive 12B	72 GB	14 tok/s4-bit quantization
Mistral Small 4MoE	119Bactive 6.5B	71.4 GB	26 tok/s4-bit quantization
Laguna S 2.1MoE	118Bactive 8B	70.8 GB	21 tok/s4-bit quantization
GPT-OSS 120BMoE	116.8Bactive 5.1B	70.1 GB	33 tok/s4-bit quantization
Sarvam 105BMoE	105Bactive 10.3B	63 GB	16 tok/s4-bit quantization
Qwen 3 Coder Next 80BMoE	80Bactive 3B	48 GB	54 tok/s4-bit quantization
Ornith 1.0 35BMoE	35Bactive 3B	21 GB	54 tok/s4-bit quantization
Ornith 1.5 35BMoE	35Bactive 3B	21 GB	54 tok/s4-bit quantization
Qwen 3.6 35BMoE	35Bactive 3B	21 GB	54 tok/s4-bit quantization
Laguna XS 2.1MoE	33Bactive 3B	19.8 GB	54 tok/s4-bit quantization
Qwen 2.5 32BDense	32.5B	19.5 GB	7 tok/s4-bit quantization
Gemma 4 31BDense	31B	18.6 GB	7 tok/s4-bit quantization
Qwen 3 Coder 30BMoE	30.5Bactive 3.3B	18.3 GB	50 tok/s4-bit quantization
Granite 4.1 30BDense	30B	18 GB	7 tok/s4-bit quantization
Granite 4.2 30BDense	30B	18 GB	7 tok/s4-bit quantization
Muse GlimmerDense	30B	18 GB	7 tok/s4-bit quantization
GLM 4.7 FlashMoE	30Bactive 3B	18 GB	54 tok/s4-bit quantization
Nemotron 3.5 LightningMoE	30Bactive 3B	18 GB	54 tok/s4-bit quantization
Fara 1.5 27BDense	27B	16.2 GB	8 tok/s4-bit quantization
Qwen 3.6 27BDense	27B	16.2 GB	8 tok/s4-bit quantization
Qwen 3.8 27BDense	27B	16.2 GB	8 tok/s4-bit quantization
Gemma 4 26BMoE	26Bactive 4B	15.6 GB	41 tok/s4-bit quantization
GPT-OSS 20BMoE	20.9Bactive 3.6B	12.5 GB	46 tok/s4-bit quantization
Instella-MoE 16B ThinkMoE	16Bactive 2.8B	9.6 GB	58 tok/s4-bit quantization
Qwen 3 14BDense	14B	8.4 GB	16 tok/s4-bit quantization
Gemma 3 12BDense	12B	7.2 GB	19 tok/s4-bit quantization
Gemma 4 12BDense	12B	7.2 GB	19 tok/s4-bit quantization
Fara 1.5 9BDense	9B	5.4 GB	25 tok/s4-bit quantization
Ornith 1.0 9BDense	9B	5.4 GB	25 tok/s4-bit quantization
Ornith 1.5 9BDense	9B	5.4 GB	25 tok/s4-bit quantization
Qwen 3.5 9BDense	9B	5.4 GB	25 tok/s4-bit quantization
Qwythos 9BDense	9B	5.4 GB	25 tok/s4-bit quantization
Granite 4.1 8BDense	8B	4.8 GB	28 tok/s4-bit quantization
Granite 4.2 8BDense	8B	4.8 GB	28 tok/s4-bit quantization
Qwen 3 8BDense	8B	4.8 GB	28 tok/s4-bit quantization
Ling 3.0 TinyMoE	7.9Bactive 1.3B	4.7 GB	114 tok/s4-bit quantization
Fara 1.5 4BDense	4B	2.4 GB	56 tok/s4-bit quantization
Gemma 3 4BDense	4B	2.4 GB	56 tok/s4-bit quantization
Qwen 3 4B Instruct 2507Dense	4B	2.4 GB	56 tok/s4-bit quantization
Qwen 3.5 4BDense	4B	2.4 GB	56 tok/s4-bit quantization
Spark X2.5 4BDense	4B	2.4 GB	56 tok/s4-bit quantization
Granite 4.1 3BDense	3B	1.8 GB	73 tok/s4-bit quantization
Granite 4.2 3BDense	3B	1.8 GB	73 tok/s4-bit quantization
Llama 3.2 3B InstructDense	3B	1.8 GB	73 tok/s4-bit quantization
Ministral 3 3BDense	3B	1.8 GB	73 tok/s4-bit quantization
Nanbeige 4.2 3BDense	3B	1.8 GB	73 tok/s4-bit quantization

1 video found Showing 1-1

Video Alex Ziskind 2026-08-26

224GB of GPU Memory on 1 Desk and It Should Not Work

Tests combining multiple GPUs into a single desktop setup with 224GB of memory to evaluate performance beyond intended specifications using Strix Halo, LLM and Nvidia components.

Testing Qwen 3.5 122B Qwen 2.5 32B RTX PRO 6000 RTX 5080 Radeon 8060S 96GB