Mixture-of-Kittens (MoK) is Cursor's new open-source CUDA megakernel for training Mixture-of-Experts (MoE) models on NVIDIA GB300 NVL72 racks. It is already running in production, powering the training of Composer, Cursor's agentic coding model, across tens of thousands of GPUs. Benchmarked against every major public baseline, it is up to 2.37x faster on MXFP8 forward passes and delivered a 1.41x end-to-end training throughput gain over Cursor's previous DeepEP-based stack.

The bottleneck nobody talks about enough

In large MoE models, each input token is routed to a small subset of "experts" (specialized feed-forward sub-networks) that may live on different GPUs. This means tokens have to travel across the network before and after computation, a process called dispatch and combine. Depending on the workload and training configuration, the MoE layer can consume more than half of end-to-end training time. The compute part of that layer had already been optimized; the bottleneck had shifted to inter-GPU communication.

Earlier work at Cursor, including custom MXFP8 and NVFP4 training kernels and a "warp decode" approach for MoE inference, optimized only the compute portion and assumed inter-GPU communication would be handled separately. In production workloads, communication had become the limiting factor. That realization forced a ground-up redesign.

The move to GB300 NVL72s added two new constraints. An NVL72 is a multi-node rack within a single NVLink domain, enabling fast, fine-grained overlap of computation and communication across all 72 GPUs. But the integrated Grace CPUs tend to be slow relative to the GPUs, so GPU streams easily caught up to CPU-side work, causing the GPU to be completely idle during that time. Any solution had to aggressively minimize CPU involvement.

What MoK actually does

MoK fuses all MoE communication and computation into a single kernel, is fully deterministic, and achieves state-of-the-art performance against publicly available implementations. The "megakernel" design means there are no kernel launch boundaries separating dispatch, expert computation, and combine. Everything runs inside one persistent kernel, with different groups of Streaming Multiprocessors (SMs) assigned to different tasks and coordinating through shared counters.