Ai4Users.comLocal AI & LLM community
Sign in to contributeSign in

LLM model: LFM 3 1.7B

Model profile
Model Size Approximate size
LFM 3 1.7BModel source: provider262,144 tokens max native context 1.7B 1 GB4-bit quantization
Compatible hardware Estimated speeds, not benchmark results: calculated from memory bandwidth and model size. Real results can differ significantly because there is no precise formula for deriving LLM generation speed from hardware specifications alone.
Hardware Memory Bandwidth vLLM Estimated generation
Raspberry Pi 4 8 GBLPDDR4 RAM 12.8 GB/s no 6 tok/s4-bit quantization
Raspberry Pi 5 16 GBLPDDR4X RAM 17 GB/s no 9 tok/s4-bit quantization
DDR4 16GB Computer 16 GBRAM 51.2 GB/s no 27 tok/s4-bit quantization
DDR5 16GB Computer 16 GBRAM 89.6 GB/s no 46 tok/s4-bit quantization
DDR5 32GB Computer 32 GBRAM 89.6 GB/s no 46 tok/s4-bit quantization
Mac mini M4 24 GBUnified RAM 120 GB/s no 61 tok/s4-bit quantization
Mac mini M6 24 GBUnified RAM 170 GB/s no 85 tok/s4-bit quantization
RTX 2000 Ada 16 GBGPU VRAM 224 GB/s yes 110 tok/s4-bit quantization
Radeon 8060S 96GB 96 GBUnified RAM 256 GB/s yes 125 tok/s4-bit quantization
M4 Pro 24 GBUnified RAM 273 GB/s no 132 tok/s4-bit quantization
DGX Spark 128 GBUnified RAM 273 GB/s yes 132 tok/s4-bit quantization
RTX 4060 Ti 16GB 16 GBGPU VRAM 288 GB/s yes 139 tok/s4-bit quantization
M5 Pro 64 GBUnified RAM 307 GB/s no 147 tok/s4-bit quantization
RTX 3060 12GB 12 GBGPU VRAM 360 GB/s yes 169 tok/s4-bit quantization
RTX 5060 Ti 16GB 16 GBGPU VRAM 448 GB/s yes 204 tok/s4-bit quantization
M5 Max 128 GBUnified RAM 614 GB/s no 265 tok/s4-bit quantization
M3 Ultra 96 GBUnified RAM 819 GB/s no 331 tok/s4-bit quantization
RTX 5080 16 GBGPU VRAM 960 GB/s yes 372 tok/s4-bit quantization
RTX 3090 Ti 24 GBGPU VRAM 1008 GB/s yes 386 tok/s4-bit quantization
M5 Ultra 256 GBUnified RAM 1200 GB/s no 435 tok/s4-bit quantization
RTX 5090 32 GBGPU VRAM 1792 GB/s yes 560 tok/s4-bit quantization
RTX PRO 6000 96 GBGPU VRAM 1792 GB/s yes 560 tok/s4-bit quantization
DGX H200 1128 GBGPU HBM3e 4800 GB/s yes 880 tok/s4-bit quantization
DGX Station 748 GBCoherent Memory 7100 GB/s yes 988 tok/s4-bit quantization
ET900N G3 748 GBCoherent Memory 7100 GB/s yes 988 tok/s4-bit quantization
DGX B200 1440 GBGPU HBM3e 8000 GB/s yes 1018 tok/s4-bit quantization
GB200 NVL72 13400 GBGPU HBM3e 8000 GB/s yes 1018 tok/s4-bit quantization
GB300 NVL72 20000 GBGPU HBM3e 8000 GB/s yes 1018 tok/s4-bit quantization
2 videos found Showing 1-2