Monitorability evaluations and policy proposals

There has been a lot of discussion recently of policy proposals around monitorability, such as Redwood's proposal and my counter-proposal . I'd like to say a bit more generally about how I'm thinking about monitorability. Note that nothing here is an official Anthropic position; these are just my personal thoughts. First, a categorization. I currently think about four different types of monitorability: Control Monitorability: the extent to which Control and monitoring techniques would likely catch catastrophic sabotage actions, like escaping a sandbox or setting up a rogue internal deployment. We can measure this via reference to sabotage evaluations like LinuxArena or any evaluation that measures the model’s capability to take dangerous actions like escaping a sandbox while looking to a monitor like it's doing some innocuous task (like a cyber eval). Training Monitorability: the extent to which reward models, monitors, and oversight in training would likely catch misaligned or undesirable actions in training, such as sneaky/subtle reward hacks. Auditing Monitorability: the extent to which alignment audits, evaluations, and test deployments would likely incriminate a systematically misaligned model (e.g. a schemer)—as well as the extent to which monitor evaluations would likely detect a model's propensity to collude as a monitor. We can measure this via reference to the model's capability to consistently distinguish between our most realistic alignment audits / monitor evals…