Alibaba’s Qwen team has released Qwen2.5-Math-PRM-7B, a 7-billion-parameter process reward model that assigns a score to each step in a candidate math solution. Developers can use those scores to rank sampled answers, guide search, or provide step-level rewards during training.

The release joins a larger 72B process reward model and Qwen’s existing outcome reward model. The outcome model assigns one score to a complete solution. The process model exposes local signals that can identify where a derivation deteriorates, while a separate policy model handles generation.

A 7B verifier in one view

One score for every step

A process reward model receives a problem and a completed reasoning chain, then emits a score at each marked boundary. Each value estimates the model’s confidence that the preceding step is acceptable in context. In Qwen’s worked example, scores of [1.00, 0.19, 0.98, 1.00] identify the second step as the likely failure point.

  • Best-of-N ranking: Generate N candidate solutions, score each chain, and select the highest-ranked answer.
  • Tree search: Score partial derivations so weak branches can be pruned before further decoding.
  • Reinforcement learning: Use intermediate scores as denser rewards than a single end-of-response grade.
  • Feedback systems: Flag a likely faulty step for a tutor, grader, or review interface.

Sentinels make the API work

Qwen’s interface depends on two delimiters: double newlines separate the written steps, and