TL;DR This work replicates and extends the findings of Arditi et al. [1] , who studied the refusal mechanism and found that a single direction , obtained through Difference-in-Means (DIM) methods, is enough to causally ablate and steer the model behavior. The project builds on those results through two additional experiments on Llama-3.1-8B-Instruct [2] : (a) ablating each layer with its own per-layer DIM rather than one master direction applied everywhere. (b) Repeat the per-layer DIM (a) but excluding layer 12 (the original master layer). Refusal is mediated redundantly across layers. No single layer is necessary, ablating every layer except 12 performs identically to ablating all 32, but layer 12’s own direction is transferable when applied everywhere on its own. The work suggests that safety alignment for the Llama3.1-8B-Instruct model, refusal is implemented as a low-dimensional, linearly accessible feature rather than something deeply entangled, which points out that safety guardrails training for the model produces a surface-level fix , rather than a deep capability change. Motivation Understanding how features are represented within an LLM has been an active area of research, one which has gained intensity concerning AI safety , as it allows us to build better safety guardrails. Mechanistic interpretability happens to work in direct relation to this by peeking into the model's internals rather than treating it as a black box , by looking at each layer's activations,…

Full article content could not be extracted automatically. Read the original below.