Local AI hardware: RTX 5080

Hardware profile
Hardware	Memory	Bandwidth	Computing power	vLLM
RTX 5080NVIDIA GeForce RTX 5080 16 GB	16 GBGPU VRAM	960 GB/s	~1801 TOPS	yes

Compatible models Estimated speeds, not benchmark results: calculated from memory bandwidth and model size. Real results can differ significantly because there is no precise formula for deriving LLM generation speed from hardware specifications alone.
Model	Size	Approx. Q4 memory	Estimated generation
GPT-OSS 20BMoE	20.9Bactive 3.6B	12.5 GB	145 tok/s4-bit quantization
Instella-MoE 16B ThinkMoE	16Bactive 2.8B	9.6 GB	176 tok/s4-bit quantization
Qwen 3 14BDense	14B	8.4 GB	60 tok/s4-bit quantization
Gemma 3 12BDense	12B	7.2 GB	69 tok/s4-bit quantization
Gemma 4 12BDense	12B	7.2 GB	69 tok/s4-bit quantization
Fara 1.5 9BDense	9B	5.4 GB	91 tok/s4-bit quantization
Ornith 1.0 9BDense	9B	5.4 GB	91 tok/s4-bit quantization
Ornith 1.5 9BDense	9B	5.4 GB	91 tok/s4-bit quantization
Qwen 3.5 9BDense	9B	5.4 GB	91 tok/s4-bit quantization
Qwythos 9BDense	9B	5.4 GB	91 tok/s4-bit quantization
Granite 4.1 8BDense	8B	4.8 GB	101 tok/s4-bit quantization
Granite 4.2 8BDense	8B	4.8 GB	101 tok/s4-bit quantization
Qwen 3 8BDense	8B	4.8 GB	101 tok/s4-bit quantization
Ling 3.0 TinyMoE	7.9Bactive 1.3B	4.7 GB	291 tok/s4-bit quantization
Fara 1.5 4BDense	4B	2.4 GB	188 tok/s4-bit quantization
Gemma 3 4BDense	4B	2.4 GB	188 tok/s4-bit quantization
Qwen 3 4B Instruct 2507Dense	4B	2.4 GB	188 tok/s4-bit quantization
Qwen 3.5 4BDense	4B	2.4 GB	188 tok/s4-bit quantization
Spark X2.5 4BDense	4B	2.4 GB	188 tok/s4-bit quantization
Granite 4.1 3BDense	3B	1.8 GB	240 tok/s4-bit quantization
Granite 4.2 3BDense	3B	1.8 GB	240 tok/s4-bit quantization
Llama 3.2 3B InstructDense	3B	1.8 GB	240 tok/s4-bit quantization
Ministral 3 3BDense	3B	1.8 GB	240 tok/s4-bit quantization
Nanbeige 4.2 3BDense	3B	1.8 GB	240 tok/s4-bit quantization

1 video found Showing 1-1

Video Alex Ziskind 2026-08-26

224GB of GPU Memory on 1 Desk and It Should Not Work

Tests combining multiple GPUs into a single desktop setup with 224GB of memory to evaluate performance beyond intended specifications using Strix Halo, LLM and Nvidia components.

Testing Qwen 3.5 122B Qwen 2.5 32B RTX PRO 6000 RTX 5080 Radeon 8060S 96GB