LessWrong AI
2026-11-16 06:56 UTC
Score 58.0
USR-0152-20261116-community-fo-81eda2a0
Full article
What sort of futures can we expect for civilization after AGI? Ideally, our future satisfies all of these properties: plausible stable desirable Surprisingly, while current visions meet some of these criteria, none fully satisfy all of them. Let’s explore some of the most commonly presented post-AGI futures. type 1: nice but unstable post-scarcity If labor is no longer scarce (because of robotics explosion), wages will collapse Resources that are currently in surplus will eventually be contested (e.g. land, energy) Even if we (humans) don't fall into a Malthusian trap, AIs (embodied or not) will replicate at a faster rate, leading to scarcity Example: The sun is our primary source of energy. We are 13 doublings away from using up all of the sunlight hitting Earth (1.7x10^17 W). And 44 doublings away from needing all of the Sun's output (3.8x10^26 W) Source : Post-AGI Civilization Talk, David Duvenaud Post-scarcity sounds appealing, but is incoherent upon inspection. laissez-faire Idea We can still have robust trade between humans and AI even when they are abundant in the physical and digital world Concretely, rich humans will want services from only humans. Example: horse-drawn carriages Problem This will necessarily be a niche market, and a tiny fraction of the total economy (to see this: consider what % of the economy is currently occupied by this category) In most of the economy, competition is wont to drive wages below human subsistence This leads to human loss of cultur…