What's in CODI's latent recurrent thoughts?

What's in CODI's latent recurrent thoughts? TL;DR CODI is a model using latent reasoning. Previous work by Cywiński et al. (2025) has found that CODI's latent recurrent states (“thoughts”) do causally carry a computation's intermediate values, in a way that can be indirectly detected with LogitLens, though the thoughts themselves remained opaque. We here ask whether it is possible use activation verbalizers (AVs: NLAs and AOs) to still decode them. To this end, we train a Qwen3-8B CODI to connect to current AVs. We find that LogitLens and J-lens read values only from late layers, but AVs recover the final answer and intermediate values well above chance, including from the thoughts themselves. This suggests that the values are in fact encoded, but requires more expressivity to decode. However, we find that editing the surfaced values, rarely steers the model output towards the corresponding counterfactual answer. GitHub HuggingFace Introduction Latent reasoning models Conventional LLMs generate their chain-of-thought through a sequence of autoregressive text tokens ("verbal CoT"). In contrast, latent reasoning models use a sequence of vectors, which can amount to a reserved vocabulary, or consist in entirely opaque vectors (see Zhu et al. (2025) for a review). Currently, verbal CoT is the major source of monitorability for a model [1] . Latent reasoning however would significantly lower the monitorability since the vector's content is not obvious anymore. Therefore, it is of…