LMSYS and humans&, in collaboration with NVIDIA, have released two Blackwell-native low-precision recipes for reinforcement learning post-training inside Miles, the open-source RL framework built on SGLang and Megatron-LM. The recipes bring end-to-end MXFP8 (8-bit microscaling) and NVFP4 (4-bit) training to the full RL pipeline, and they are fully open-sourced across every layer of the stack.

Why Blackwell changes the math

Previous low-precision RL work used Hopper-era FP8 recipes, where scaling factors were computed in software around the Tensor Core path. Blackwell hardware changes that equation dramatically. The blog normalizes NVIDIA HGX platform dense Tensor Core specs to per-GPU throughput, showing that a B200 delivers 2.25 PFLOPS at BF16, 4.5 PFLOPS at FP8, and 9 PFLOPS at FP4. Going to FP4 is not a marginal gain; it is a 4x compute multiplier over BF16 on the same chip.

Microscaling (MX) formats in NVIDIA's latest Blackwell GPUs represent a major leap in enabling precision scaling, combining narrow floating-point data types with per-block scaling factors for a fine-grained approach to quantizing tensors. The key difference from older software-scaled FP8 is that Blackwell handles the rescaling natively in hardware, which is what makes MXFP8 and NVFP4 worth targeting specifically.

The core problem: keeping rollout and training in sync

In low-precision RL, rollout, training, checkpoint conversion, and live weight updates must agree on one precision contract, or the sampler and trainer policies will diverge. This is the central systems challenge. If the model generating samples and the model being trained are quantized differently, you are effectively training on data from a different policy, which destabilizes learning.

Reinforcement learning workloads differ from pretraining in a crucial way: rollout generation dominates compute. Modern RL training may spend 70-90% of GPU time generating long sequences across thousands of parallel environments. That makes rollout the primary bottleneck, and the primary target for precision optimization.

Recipe 1: End-to-end MXFP8

MXFP8 is a microscaling format where every 32 consecutive E4M3 values share one local E8M0 scale, and the block is one-dimensional. The MXFP8 recipe covers rollout, forward propagation, weight-gradient GEMMs, and data-gradient GEMMs, while selected tensors remain BF16 through precision-control rules. This is the most direct Blackwell-native extension of the earlier FP8 RL work from LMSYS.