Value Generalisation 3: Pre-aligned AIs

When we get explicit strong generalisation to work (see the first post on the matter and the second ) my dream would be to create pre-aligned generalising AIs. Think about the usual conflict between alignment and capabilities, between doing the right thing and doing the easy thing. The standard narrative puts the good people at a constant disadvantage: they have to carefully plan every AI advance, always on the lookout for potential dangers. While those who don’t care can just YOLO and let it rip and let their AIs get ever more powerful without taking any responsibility. Now, in reality, there is some nuance to the story; but I don’t want to nuance it, I want to turn it on its head. I want to create AIs so that the good people can YOLO and reap the rewards of increased AI capabilities. While the bad actors have to carefully plan and limit their AIs and constantly restrict what the AIs can do. Pre-aligned AI: binding morality to empirical concepts A pre-aligned AI is an AI whose morality increases with its capabilities. The core idea is simple. Start by designing an AI capable of value generalisation and of empirical generalisation. It’s an AI that can learn and improve its world model and capabilities, both empirical and moral. Its moral goals will be defined in terms of concepts, with these initially defined themselves by simple terms in its starting empirical world model. And it will act on these goals. But, initially at least, the simple concepts won’t be very robust and…