Alibaba’s Qwen team has published official quantized builds of Qwen3-8B on the model page. The repository provides Q4_K_M and Q8_0 GGUF files for llama.cpp, recent Ollama releases, LM Studio, and compatible runtimes. Developers can run the same checkpoint in an extended reasoning mode or a faster direct-answer mode without converting the original weights.

GGUF removes a conversion step

GGUF packages model weights and runtime metadata in a format designed for local inference. Quantization stores weights at reduced precision, lowering storage and memory requirements while introducing a potential quality tradeoff.

Total runtime memory exceeds the file size because inference also allocates a key-value cache, compute buffers, and prompt state. With a default 16-bit KV cache, a 131,072-token window for this architecture can consume roughly 18 GiB for the cache alone, before weights and other buffers. KV-cache quantization and partial CPU offload can reduce VRAM pressure, with corresponding speed or quality costs.

Reasoning becomes a request-level switch

Qwen3-8B supports thinking and non-thinking behavior within one checkpoint. Thinking mode emits intermediate reasoning inside <think> blocks before the final response, while non-thinking mode answers directly with lower token use and latency.