Prism ML has released Ternary-Bonsai-2-9B, a quantized vision-language model derived from Qwen3.5-9B. Its default GGUF compresses the roughly 8.95-billion-parameter language backbone from about 17.9 GB at FP16 to 2.42 GB, while the vision components ship separately. The model repository showed more than 896,000 downloads and 3,700 likes at the time of writing, though Hugging Face totals can include repeated and automated downloads.

The smaller weight files expand the hardware available for local inference, including laptop GPUs, Apple silicon and CPU-only systems. Several constraints shape deployment: the smallest formats require Prism ML’s llama.cpp fork, runtime memory exceeds the GGUF file size, and using the full 262,144-token context window adds substantial memory and latency.

How 9B parameters fit into 2.42 GB

The language model stores its major weight matrices with ternary values from {-1, 0, +1}, paired with a shared FP16 scale for each group. The scheme covers embeddings, attention projections, MLP projections and the language-model head. Custom kernels read the packed representation during inference, avoiding a full FP16 copy of the weights.

A blockwise Hadamard rotation is folded into the stored weights, and the runtime applies the corresponding transform to activations. The rotation spreads large outlier values across dimensions, reducing the quantization error caused by mapping weights into three values.

The default PQ2_0 file is about 7.4 times smaller than the 17.9 GB FP16 language model. That ratio applies to the language weights on disk; it excludes the optional vision package, context state, compute buffers and runtime overhead.

A 262K window with practical limits