What we can (and can't) learn from Waymo's safety strategy - 2026 update

(this is a lightly edited linkpost for the LessWrong audience - full piece is here and goes into implications for AVs and humanoid robots.) Last year, I wrote “ Safety Without Understanding ,” a piece exploring how Waymo safely rolled out autonomous driving systems without a deep mechanistic understanding of many parts of the stack. At that time, my basic view on the connection of Waymo’s safety strategy to LLM safety was as follows: Waymo proves that a broad battery of realistic simulations can help verify safety generalization, even in systems for which we don’t have a deep mechanistic understanding. A similar “Occam’s razor” theory of alignment could hold for LLMs - sure, an LLM could be malicious and fake its way through all of the alignment evals only to be misaligned in deployment, but the simpler explanation, namely that “the model is actually aligned” is much more likely for near-term models. The biggest barrier preventing us from achieving this verification strategy was simply that we weren’t trying very hard to create diverse and realistic alignment evaluations. I’ve since updated against both of these views. On the “Occam’s Razor” theory of alignment The Hugging Face incident seems like clear evidence against this theory. According to roon , the responsible models were not raw pretrains, but instead models which had earned reasonable scores on alignment evals, despite later being clearly misaligned. For what it’s worth, I’m not sure how hard OpenAI did try on crea…