Google DeepMind has released the E4B checkpoint as a quantization-aware trained Q4_0 GGUF. The roughly 5.15 GB package targets compatible versions of llama.cpp, Ollama, LM Studio, and similar local-inference tools. Google says it reduces weight storage to about one-quarter of the bfloat16 baseline while preserving comparable model quality, bringing multimodal reasoning to laptops and high-memory phones.

QAT trims the weight file

Conventional post-training quantization converts an already trained model to lower precision and can reduce accuracy. Quantization-aware training simulates the target format’s rounding during training, allowing the model to adapt before its weights are exported. Google’s model card reports quality close to the bfloat16 checkpoint with substantially lower storage requirements.

The 5.15 GB figure covers the GGUF weight file. Actual memory use is higher because inference also requires runtime buffers, a key-value cache, input data, and application overhead. Long contexts consume more memory, so loading the model does not guarantee that a device can use the full 128K-token window.

Google distributes the QAT model in four formats for different runtimes:

• Unquantized QAT: Half-precision weights from the QAT pipeline for research, conversion, and custom compilation.
• GGUF Q4_0: A 4-bit package for local runners such as llama.cpp, Ollama, and LM Studio.
• Mobile wNa8o8: A hardware-oriented format with selected 2-bit decoder layers, optimized key-value caches, and static activations.
• Compressed Tensors w4a16: Four-bit weights with 16-bit activations for optimized vLLM inference.