Extreme concentration of power over ASI has non-obvious advantages

This post is an extension of a collaboration with cousin_it on the question How risky would it be if powerful AI obeyed one or a few people?. There he argues for a common position: a future controlled by one or a few humans with powerful AI aligned to their intent is likely to produce terrible outcomes. My position is guardedly optimistic, [1] for reasons I think are fairly novel: humans tend strongly to be better and become better over time under good circumstances, and near-perfect power and knowledge are the best circumstances. That post contains his essay and the abstract and overview sections of this post as my shorter response. This piece grew longer than our original target, because the subject is potentially critical for alignment strategy, and has not been analyzed in any depth, to my knowledge. Abstract: Concentration of power over AGI/ASI seems quite possible. The first AGIs being aligned to intent (or instructions) over values seems fairly likely . So one or a few individuals or small groups gaining power over ASI seems fairly likely. [2] Thus it seems relevant to technical alignment strategy (value alignment vs. corrigibility) to worry about what individuals might do with such immense power. Here intuitions diverge, and careful analysis is scarce. When we imagine one or a few people in charge of the whole future, it's intuitively very scary. We imagine a future serving the values of current and historically powerful people, which typically range between lacking…