The framework addresses high autoregressive decoding costs by employing differentiated training strategies tailored to task types, including a lightweight multi-token prediction mechanism for open-ended dialogue and a dedicated block diffusion model for code generation and mathematical reasoning.

During training, AngelSpec uses a shared-parameter scheme that unrolls predictions and relies on generated rollouts rather than reference tokens, while the target model backbone and output head remain frozen to prevent degradation of the final model’s output quality.

Tencent has open-sourced AngelSpec, a speculative-decoding framework designed to improve LLM inference across multi-token prediction and block-parallel architectures, with open-weight pre-trained draft model checkpoints available on Hugging Face and ModelScope for Tencent’s Hunyuan models.

AngelSpec is an open-source unified training framework from Tencent aimed at dramatically improving inference throughput for large models in real-world scenarios.

Tencent released AngelSpec as an open-weight speculative-decoding framework designed for multi-token prediction and block-parallel architectures, with seven pre-trained draft model checkpoints available on Hugging Face and ModelScope for Tencent’s Hunyuan models.

Training optimizations include compiling rollout operations into optimized kernels to minimize memory usage, maintain a memory footprint near a standard processing pass, and enable long-context training through sequence parallelism.

The framework introduces DFly, an enhanced block-diffusion drafting architecture with a hybrid target-conditioning backbone and per-layer residual fusion weights, plus a small sequential head to avoid acceptance decay without excessive computation.

A novel block diffusion framework named DFly is introduced to optimize feature utilization and intra-block dependencies, enabling high-throughput parallel generation through a hybrid target conditional encoding backbone and a previous conditional autoregressive head.

AngelSpec introduces DFly, a hybrid block-diffusion drafting architecture with a shared conditioning backbone and per-layer residual fusion weights, plus a small sequential head to prevent acceptance decay efficiently.

A disaggregated system architecture captures internal state data from inference workers and transfers it to training nodes via high-speed networks, enabling long-context training through sequence parallelism and document-aware packing mechanisms.

AngelSpec uses a disaggregated system architecture where inference worker processes capture internal state data and transfer it to training nodes via high-speed networks, enabling long-context training through sequence parallelism and document-aware packing; training efficiency is further boosted by compiling rollout operations into optimized kernels to keep memory usage close to a standard processing pass.

AngelSpec incorporates a dynamic verification mechanism that adjusts verification depth at runtime based on online load, hardware conditions, and prefix confidence to balance computational resources and throughput.