QuantTrio releases a 4-bit AWQ build of Qwen3.6-35B-A3B

QuantTrio has published an AWQ repository for Qwen3.6-35B-A3B, Alibaba’s open-weight mixture-of-experts model. The 4-bit release cuts the weight files from roughly 70 GB in BF16 to about 24 GiB, making deployment practical on a multi-GPU workstation, a pair of 48 GB cards, or a comparable server.

The repository has recorded more than 788,000 downloads on Hugging Face. That counter measures file requests rather than unique users or completed deployments, but it indicates substantial early interest in a version that requires less memory than the original weights.

The base model card describes the first open-weight release in the Qwen3.6 family. It contains 35 billion total parameters while activating about 3 billion for each token, and Alibaba distributes it under the Apache 2.0 license.

Four-bit weights cut the deployment cost

AWQ, short for Activation-aware Weight Quantization, stores supported linear-layer weights at 4-bit precision while retaining higher precision where the runtime requires it. This reduces weight memory and bandwidth, with a possible quality loss compared with BF16.

QuantTrio generated this build with a data-free quantization process, so it did not run a calibration corpus through the model to choose quantization scales. That removes calibration data from the build pipeline, although a carefully selected calibration set can sometimes produce better accuracy.

The 24 GiB figure covers stored weights. A running server also needs memory for the KV cache, activations, CUDA graphs, temporary buffers, and concurrent requests. A 24 GB card therefore cannot be assumed to hold the complete service merely because the files are close to that size.