Alibaba’s Qwen team has released Qwen3.5-0.8B-Base, the smallest checkpoint in a family that reaches 397 billion parameters. The 0.8B model carries the family’s native image, video, and text support, along with a hybrid attention stack designed to reduce inference memory during long sequences.

The Apache 2.0 checkpoint is a pretrained base model with no instruction tuning. Raw generations may follow prompts inconsistently, so practical deployments will usually require supervised fine-tuning, preference tuning, or a community instruction-tuned derivative.

Full stack, 0.8B parameters

Native multimodal support means the checkpoint includes the visual processing path used during training. A visual encoder converts images and video into representations consumed by the language stack, giving developers one model interface for media and text tasks.

The 24-layer causal language model repeats the same four-layer pattern six times: three Gated DeltaNet blocks followed by one conventional attention block. Each block also contains a feed-forward network. Multi-token prediction adds training objectives for several future positions, providing a denser learning signal than next-token prediction alone.

The small parameter count reduces weight memory and makes adapter training more accessible. Actual hardware requirements still depend on numerical precision, sequence length, media resolution, optimizer state, batch size, and whether optimized DeltaNet kernels are available.