Speculative decoding has become one of the most practical tools for squeezing more throughput out of large language models without touching the underlying weights. The idea is simple: a small, cheap draft model proposes several tokens at once, and the full target model verifies them all in a single forward pass. If the drafts are good enough, you get multiple tokens for the cost of one. The hard part is training a draft model that's fast, accurate, and works at production scale. That's exactly what Tencent's Hunyuan team just open-sourced.

AngelSpec is a torch-native, end-to-end framework for training speculative decoding draft models, covering both the training pipeline and the deployment integration. It's not a research prototype: the framework was built to serve Tencent's own Hunyuan 3 (Hy3) model in production, a 295B MoE with only ~21B active parameters per forward pass.

The numbers that matter

On Hy3-A21B, DFly delivers the highest throughput across all concurrency levels (4 to 64) and all six benchmarks, with a 1.98 to 2.40x average speedup over the autoregressive baseline, peaking at 2.86x on code and math tasks, and running 10.5 to 11.8% faster than DFlash. The gap in accepted token length is even more telling: DFly reaches a mean accepted length of 4.79, which is +30% over DFlash (3.69) and roughly 1.6x over MTP (3.00), going up to 5.52 on HumanEval.

Accepted length is the key metric here. It measures how many draft tokens the target model accepts on average before rejecting one. A higher number means fewer full forward passes, which translates directly to throughput.

Six draft architectures, one training pipeline

The most unusual thing about AngelSpec is its breadth. Rather than committing to a single drafting strategy, the framework supports six distinct architectures, all trained through the same pipeline with a config change: