This work was done as part of the Second Look Fellowship by Arav Dhoot and supervised by Yixiong Hao and Zephaniah Roe. I'm grateful to Harshul Basava and Vanessa Ng for their feedback. This is an extension to a prior replication which can be found here . Introduction and Motivation In an earlier post , I showed that the “necessity effect” of Emmons et al . replicates across eleven models, where LLMs readily follow simple hints, even incorrect ones, but when hints require actual computation, the models are forced to verbalize that reasoning within their chain-of-thought (CoT) traces. However, that study evaluated fully trained models meant for deployment. In this post, I trace the emergence and trajectory of these behaviors across the training lifecycle. Using OLMo-3 as a candidate model, I analyze four distinct public checkpoints: pretrained, Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Reinforcement Learning from Verifiable Rewards (RLVR). This post offers a proof of concept that post-training stages directly alter (and in some cases corrupt) CoT faithfulness. It serves as concrete evidence that alignment recipes affect safety properties in unexpected, non-monotone ways. Understanding these shifts is essential if we want to design safer post-training pipelines from first principles rather than treating alignment as a black box . Experimental Setup & Controls I used the simple hint [1] injection strategy outlined in Chen et al . An incorrect hint…

Full article content could not be extracted automatically. Read the original below.