A community release on Hugging Face applies the Heretic abliteration tool to UkisAI’s Swift 1.5 Qwen3.8-27B, reducing some refusal behavior while retaining llama.cpp support for vision and multi-token prediction. The maintainer distributes the result as a 12.12 GB, single-file IQ3_S GGUF plus a 0.93 GB vision projector. The reported configuration uses roughly 25 GB of VRAM with a 262K context setting, placing it within the capacity of a 32 GB GPU.
GGUF is llama.cpp’s format for storing and running quantized models locally. Heretic modifies internal activation directions associated with refusals, avoiding a full retraining run. The release documents how those edits affect safety behavior, benchmark scores, quantization quality, and speculative-decoding speed.
One model file, one vision projector
The model card identifies the following components and runtime features:
- Model weights: 12.12 GB in IQ3_S format, averaging 3.55 bits per weight.
- Vision projector: a 0.93 GB F16 file built from Swift’s vision tower, which maps image features into the language model.
- Speculative decoding: the original MTP head is included so llama.cpp can draft and verify multiple tokens.
- Quantization data: the build uses UkisAI’s tensor allocation and importance matrix, which guide how much precision each tensor retains.
- Runtime support: the model card covers llama.cpp, LM Studio, Jan, Ollama, Docker Model Runner, and vLLM.
UkisAI’s Swift Open License applies to the derivative. Organizations with at least US$1 million in annual revenue must obtain a separate Swift Enterprise License.