Inception in DiffusionGemma - Jailbreaking a Diffusion Language Model by Pinning Tokens Anywhere on the Canvas

Authors: Theresa G., Simon S. , Siva Kumar Lakkoju . Epistemic status/effort : exploratory red-teaming as part of a two-day hackathon during ARENA 8.0 . Our attacks can be reproduced based on our GitHub Repo (here) . The interpretations are mostly intuitions, as the evaluation was small. Hence, treat the framing as "this attack surface is worth taking seriously" in the context of diffusion models moving into production for things like inline editing. TL;DR Auto-regressive LLMs generate from left to right, a property exploited by so-called pre-fill jailbreaks ( Li et al. ). Those attacks involve pinning an (adversary) sequence at the opening of the model’s response, and the model has to continue from there. In contrast, diffusion models such as DiffusionGemma denoise a whole canvas of (noisy) tokens in parallel with bidirectional attention rather than left to right. This removes the adversaries' constraint to only pin sequences at the start and enables attackers to pin tokens in the middle as well as at the end of a sequence. A sequence can also be pinned "softly", where we condition the model on a sequence with a low assigned probability, so that it can overwrite the tokens as part of its denoising process. This conditioning nudges the model toward a response pattern early, and in practice, it often keeps latching onto that pattern even when it could, in principle, drop it. Results : Pre-fill attacks are not unique to diffusion, as DiffusionGemma (harm score , , where 0 is b…