An independent developer has released Qwen-2.5-1B-RLCD, an Apache 2.0 inference engine for structured classification and extraction on Apple Silicon. The engine uses MLX to score multiple JSON fields in parallel against a shared key-value cache, avoiding the sequential token generation normally required to spell out an object. It requires no fine-tuning and relies on the model’s existing logits, which are its scores for possible next tokens.
The implementation targets schemas whose values come from fixed sets, such as booleans and categorical enums. The project name references “1B,” while the published configuration and benchmarks use mlx-community/Qwen2.5-1.5B-Instruct-4bit.
Why JSON decoding drags
Conventional JSON mode and grammar-guided decoding remain autoregressive: each generated token depends on the previous tokens. Every step invokes the model again and reads from its attention cache, so latency rises with the length of field names, values, punctuation, and other serialized output.
A schema with 28 fields can therefore require hundreds of sequential decoding steps, even when every value belongs to a small set of known labels. Constrained decoding can block invalid tokens, but it still generates the object token by token.
Qwen-2.5-1B-RLCD treats each field as a classification problem. Given a predefined candidate set, the engine scores the permitted values and selects the highest-ranked option. Programmatic assembly then produces the JSON object without asking the model to generate braces, keys, commas, or quotes.