Your speculations about capabilities are not automatically safe

Thank you to @Thane Ruthenis for discussing this topic with me previously. Some arguments in this essay are taken from that discussion, but all mistakes and/or bad arguments should be attributed to me. You can view the original discussion here (optional for reading this essay). Thank you as well to @grainsilo for feedback on an earlier version. Consider the following argument for discussing AI capabilities as a safety-focused person: Understanding AI capabilities better has many strategic benefits, particularly for timelines. Yes, good ideas tend to break containment, so some of our AI capabilities analysis may get read by capability researchers…but at the same time, these people have already thought about these questions more deeply and thoroughly than we have. Therefore, none of our arguments are new to capability researchers. Therefore, we will have no impact on advancing AI capabilities or timelines by discussing this topic. I’ve seen this argument made explicitly in some places , I presume it's made in many people’s heads whenever they post about capabilities, and I’ve certainly made this argument to myself before. I now also think this argument is wrong. I don’t really know what to do about this conclusion. Maybe the answer is nothing, but I don't know that, so it seems like a good idea to make people more aware of the reasons why I think this, in the hopes that it prompts further discussion, reflection, and caution. Problem 1: High-Surface-Area Questions Certain field…