Qwen3-VL 8B gets official GGUF weights for local multimodal inference

Qwen has published official GGUF weights for Qwen3-VL-8B-Instruct, its 8-billion-parameter vision-language model. The release lets developers run image, video, and text workloads through common local inference tools without assembling a Transformers environment and custom vision preprocessor.

GGUF packages model tensors and metadata for runtimes such as llama.cpp and Ollama. Qwen separates the language model from the multimodal projector, or mmproj, which contains the vision components needed to convert images and video into representations the language model can process.

Pick the precision per component

The listed language-model sizes exclude the projector, KV cache, image or video buffers, and runtime overhead. A Q4_K_M model paired with a quantized projector can fit within many 16 GB unified-memory systems at moderate context lengths. Memory use rises sharply as context grows, making the full 256K-token window impractical on that hardware.

Separate files allow combinations such as a 4-bit language model with an F16 projector for OCR and grounding tests, or quantized files for both components when memory is constrained. Quantization can change recognition and localization accuracy, so production configurations should be evaluated on representative documents, screenshots, and video.