Plan A, by AI-2040

The folks who wrote AI 2027 have written a more optimistic narrative , which focuses more on hopes for good policies than on predictions about what policies we’ll get. Plan A’s narrative seems halfway between a science fiction story and a proposed treaty. Like most science fiction, I expect it to err in the direction of describing the world as more human-understandable and relatable than what we’ll actually get. The broad outlines come close to the scenario that I analyzed in Financial Costs of an AI Pause? , which is what I predict that fairly competent governments would do. AI-2040 adds much more detail than I was able to provide, some of it surprising. The devil is in the details. I largely endorse their advice. The rest of this post will focus on many small doubts about their advice and their predictions about what that advice would produce. Keep in mind that this is just a plan. Expect plans to change in response to contact with reality. Rates of AI capability growth ought to change in response to better evidence about the difficulty of alignment. Please read the Insider Perspective section. It’s a little more technical, but it answers several nontechnical questions that were covered inadequately in the main story. What Success Looks Like The long-term vision of Plan A is to hand control over to AIs that we’re pretty sure have our interests at heart. The authors guess that will happen in 2040. Plan A’s vibes suggest the handover will go well, but the authors seem carefu…