The August Risk Report describes the 2x Acceleration Threshold as follows:
“The risk threshold set out in our RSP—a doubling of the pace of progress beyond pre-AI-acceleration rates, attributable to automation of AI R&D—functions as a potential early warning, rather than evidence that the threat has already materialized.”
They do not share a precise threshold that would indicate risk was high, but their threat model describes illustrative scenarios:
“The threat: We believe that AI models could, in the next few years, have a broad range of capabilities that exceed human capabilities. In particular, most or all of the work needed to advance research and development in key domains—from robotics to energy to cyberwarfare to AI R&D itself—may become automatable.
If this is correct, there could be enormous benefits, but these benefits would come with several distinct but related concerns:
- Sufficiently rapid progress in some areas of technology could change power structures both within and between nations. For example, if an authoritarian regime built a self-reinforcing lead in AI over the rest of the world, this lead could compound into a major advantage in science and technology—which could in turn translate to very sudden and disruptive advances in the ability to wage war, surveil citizens, and/or wield extremely powerful technology over the long term.
- If AI models develop dangerous goals while heavily automating R&D in key domains, they may cause unbounded harm—up to and including humanity losing control over civilization entirely—by leveraging novel technology and their access to it.
- If AI can be used to automate AI R&D itself, the result may be extreme acceleration in AI progress, putting a very broad set of risks on the table, including but not limited to those discussed in this report.”
Based on these descriptions and conversations with Anthropic staff, we understand the overall risk designation of “low/medium/high” as being about the risk that models at the time directly enable the kinds of catastrophic concerns described above, not the risk that they cross their RSP threshold (though we believe this could have been made clearer in the report).