x

This is a linkpost for https://alignment.anthropic.com/2026/automated-em-search

Automating subtle emergent misalignment research

7draganover

8evhub

5

AI

Frontpage

New Comment

Thanks! I'm pretty surprised that this seems to be a similar hill-climbing scaffold to the one from the automated weak-to-strong research post ~6 months ago? In particular, I find the following two things intrinsically at odds:

- Bullishness on automated research being on the critical path to alignment

- Prioritizing hill-climbing settings for automated alignment research (at least publicly)

There is a belief among large parts of the 3rd party safety community that alignment requires the kind of research which cannot easily be hillclimbed. I.e., it is not always clear what metrics one should climb. And, even if one obtains good scores on the metrics, it's not clear how this corresponds to improved alignment. I'm curious whether you disagree with this assessment?

To ground this in specific examples, we've used similar scaffolds to the one described in this post. While these can be useful for hill-climbing-shaped work, we find that they do not provide uplift on the diverse set of tasks that holistic alignment research seems to require.

I used to be a big proponent of the position that automating alignment would require fuzzy alignment taste that we would have to train into models explicitly, but I've recently updated away from that position and I think it now looks much more likely to me that hillclimbing concrete metrics will be more important. I'll probably write more about this soon, but for now:

- I think this research provides a very good example of how we can set up a concrete hillclimbable task and then get useful and interesting insights out the other end, not just number going up in an inscrutable way. If you look at the data here, a lot of it does make sense once you think about it, e.g. making use of implicit power-seeking vibes in descriptions of evolutionary processes.

- I also just generally think way more different alignment objectives can be converted into auditing games than is currently appreciated. Auditing games are really powerful! E.g. I think all of interpretability can be made concrete that way. And all you need to make that work is an automated pipeline for generating model organisms, which is exactly what this research is!

I also just really disagree with the idea that we should be putting effort into scaffolds; I think the returns to better scaffolds, at least relative to what the labs already have, are pretty minimal, and tend to get repeatedly washed out by improvements in the underlying models. Working on scaffolds is just not a very bitter-lesson-pilled thing to do; in my opinion, the right approach is to focus on metrics and verification processes, which scale much better, and is what we did here.

Authors: Oscar Balcells Obeso, Richard Qi, Monte MacDiarmid, Evan Hubinger

Abstract

Twitter thread

Read the full blog post here!