Recently I read Julian Mendel's article, "Mechanistic Interpretability: Peeking Inside an LLM" on Towards Data Science(https://towardsdatascience.com/mechanistic-interpretability-peeking-inside-an-llm/), and I became fascinated by the idea of moving beyond treating Large Language Models as "black boxes."
Summary of the article's premise: The post proposes that we can understand and even manipulate an LLM’s behavior by examining its internal architecture—specifically the residual stream, attention heads, and MLP layers. It treats the model as a circuit that can be reverse-engineered to see how information is processed and stored.
Key examples mentioned:
World Models: How models represent internal states of games like Chess or Othello.
Induction Heads: Specific attention heads that allow for in-context learning.
Superposition: The phenomenon where neurons are "polysemantic," representing multiple concepts at once.
Steering Vectors: Using activation addition to modify a model's behavior (e.g., making it more honest or preventing refusals) without retraining weights.
Conclusion of the article: The author concludes that while manual mechanistic analysis provides deep insights into safety, reliability, and human-like cognition, the field is rapidly moving toward automated interpretability to handle the sheer scale of modern models.
The Question: As someone looking to transition from a general understanding of Transformers to actually performing research or experiments in Mechanistic Interpretability:
- What are the canonical papers or "must-read" sequences for this field (beyond the original Transformer paper)?
I am particularly interested in resources that bridge the gap between high-level theory and Python implementation.