Why Unsloth’s 1B Llama mirror keeps spreading

Unsloth’s Llama repository has climbed Hugging Face’s trending charts. At publication, the page showed more than 800,000 downloads in its displayed monthly count and 3,600 likes. The weights come from Meta, but Unsloth packages them for easier downloading, quantized inference, and single-GPU adapter training. That distribution work explains the repository’s reach.

Small weights, broad reach

Meta released Llama 3.2 in September 2024 with 11B and 90B vision models and compact 1B and 3B text models. According to the Meta release, the smaller models target mobile devices, edge hardware, retrieval, summarization, and multilingual assistants.

The 1B Instruct model contains about 1.23 billion parameters and was produced through pruning and knowledge distillation from larger Llama models, followed by instruction tuning. Meta reports pretraining on as many as 9 trillion tokens, a December 2023 knowledge cutoff, and official support for English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai.

Grouped-query attention reduces the memory and computation required during generation by letting multiple query heads share key and value heads. The architecture supports a 128K-token context window, although usable length depends on available memory, inference software, and workload.

One model, several targets

Unsloth preserves the full BF16 safetensors weights, which occupy about 2.49 GB, and publishes sibling repositories for other runtimes. Its GGUF builds support llama.cpp, Ollama, LM Studio, CPU inference, and Apple Silicon. The separate 4-bit build targets CUDA systems and QLoRA training.

Quantization stores weights with fewer bits. It reduces storage and memory use, with some loss of numerical precision and potentially model quality. The available packages cover several deployment needs: