AI safety prizes

Rather than paying up front for AI safety research (push funding), perhaps we should pay after the fact for the work that made the most progress (pull funding). This way, you only pay for work that was actually valuable. [1] When we know what the target is, but not how to get there or who is best placed to solve the problem, a prize is a useful incentive structure. Benefits of prizes Prizes work well to incentivise innovation when: The eventual winner is unpredictable (otherwise just fund the obvious choice in advance). Trying to decide who is most likely to make a breakthrough (and therefore who to fund) is often very difficult, particularly since applicants have private information that might be hard to credibly signal. A valid solution is easily verifiable , to save time and controversy when allocating the prize. In some domains (e.g. mathematical proofs) verification is far easier than generation. Whereas in e.g. fuzzy policy work, deciding an idea is good is harder than coming up with the idea. You want, and can get, fanfare. If a prize carries a big reputational boost as well as just $, then you can incentivise a lot more effort than the raw $ would justify. Conversely, in sensitive areas, public prizes are a poor fit. Historically, prizes have worked well in e.g. DARPA’s autonomous vehicle challenges to source diverse talent into the field. Costs of prizes There are also some risks and downsides of prizes to be aware of: If you make a large prize with bad win conditio…