Prism ML has released Ternary-Bonsai-2-2B as GGUF downloads for llama.cpp. The model compresses most transformer weights to three values and reduces the language-model file from roughly 3.8 GB at FP16 to 0.52 GB in the default packing. Compatible builds can run it through CUDA, Metal, or CPU backends, making the release relevant to developers building local multimodal tools under tight storage and memory limits.

Three values replace FP16 weights

Each quantized transformer weight becomes -1, 0, or +1. This ternary representation is often called 1.58-bit quantization because three possible states contain about 1.58 bits of information. Actual files require additional metadata, so their effective storage cost is higher.

The quantization covers embeddings, attention projections, MLP projections, and the tied language-model head. Each group stores an FP16 scale that restores an approximate magnitude during matrix multiplication. Normalization weights and the linear-attention state path remain in F32 to reduce numerical instability.

A blockwise Hadamard rotation changes the basis in which the weights are stored. The runtime applies the corresponding transform to activations during inference, which explains why these files require explicit support in llama.cpp rather than a generic low-bit loader.

Choose the packing for the runtime

The three downloads contain the same ternary weights but use different storage and unpacking layouts. GGUF packages model weights and metadata in the format consumed by llama.cpp.