LessWrong AI
2026-08-03 14:21 UTC
By Stuart_Armstrong
USR-0152-20260803-community-fo-e8abb2c0
II. Anthropic reasoning with duplication is not consistent with probability properties
tl;dr There is an impossibility result in anthropic probability: no reasonable probability theory can stay consistent across a duplication event. In particular, it must violate either Bayesian updates-from priors in non-anthropic situations, or the martingale condition – today’s probabilities are expectations of tomorrow’s probabilities. In this post, I’ll extend beyond basic anthropic problems and see what happens when duplicates or copies are allowed. I’ll start with an impossibility result that may help clear up some of the confusion in anthropic probability: namely that no reasonable probability theory can stay consistent across duplication events. It’s the duplication event that is the issue. The presence of duplicates or copies is not a problem. Giving birth or creating new agents is not a problem. But duplicating an already existing agent breaks anthropic probability. That being said, just because no anthropic probability theory is perfect, doesn’t mean that some aren’t better than others. SIA is a top candidate for an anthropic probability theory (and indeed it is consistent before and after duplication events). In the final post in the series, we’ll see some of the issues with SIA and infinity, and I’ll introduce D-SIA, distributional SIA, which fixes many of the issues. Inconsistency of probability across duplication To illustrate, I’ll be using yet another variant of the Sleeping Beauty problem . In the traditional problem, a coin is secretly tossed, and Sleeping…
tl;dr There is an impossibility result in anthropic probability: no reasonable probability theory can stay consistent across a duplication event. In particular, it must violate either Bayesian updates-from priors in non-anthropic situations, or the martingale condition – today’s probabilities are expectations of tomorrow’s probabilities. In this post, I’ll extend beyond basic anthropic problems and see what happens when duplicates or copies are allowed. I’ll start with an impossibility result that may help clear up some of the confusion in anthropic probability: namely that no reasonable probability theory can stay consistent across duplication events. It’s the duplication event that is the issue. The presence of duplicates or copies is not a problem. Giving birth or creating new agents is not a problem. But duplicating an already existing agent breaks anthropic probability. That being said, just because no anthropic probability theory is perfect, doesn’t mean that some aren’t better than others. SIA is a top candidate for an anthropic probability theory (and indeed it is consistent before and after duplication events). In the final post in the series, we’ll see some of the issues with SIA and infinity, and I’ll introduce D-SIA, distributional SIA, which fixes many of the issues. Inconsistency of probability across duplication To illustrate, I’ll be using yet another variant of the Sleeping Beauty problem . In the traditional problem, a coin is secretly tossed, and Sleeping…
Full article content could not be extracted automatically. Read the original below.
Source:
LessWrong AI
· lesswrong.com