LLMs can learn (some) multi-step reasoning on filler tokens

Research update of a project done as part of MARS 5 and the Meridian Visiting Researcher Programme . Filler-token reasoning is the ability to use extra tokens (such as a chain of thought) to reason about a task without using the words emitted at those token positions. This is possible because attention can move information between token positions without routing it through the words the LLM emits. This can give a model more parallel, but not more serial, computation. The canonical demonstration of this is the "Let's Think Dot by Dot" paper ( Pfau, Merrill & Bowman, 2024 ). There, the authors train models to solve completely parallelizable (TC⁰) tasks with chains of thought made only of "." tokens. Our contributions: We change Pfau et al.'s training method so that models can learn multi-step reasoning on filler tokens. We demonstrate this on two models and three out of seven multi-step reasoning tasks. We provide a theoretical analysis of how many layers a transformer needs to solve each of our multi-step reasoning tasks, depending on the length of the task. We measure empirically how many reasoning steps we can train Qwen3.6-35B-A3B and gpt-oss-20b to do on filler tokens for each task. Overall, this project has informed which side tasks we use to study steganographic reasoning. We also hope this writeup shows that filler-token reasoning is not confined to fully parallelizable tasks. Motivation Visualizing different reasoning modes. Open reasoning: each intermediate result is…