Research Priorities for Multi-Agent Swarms

Multi-Agent swarms are important for two reasons: They are probably much faster and somewhat more capable than single agents (Capability) Multi-agent alignment is a different beast than single-agent alignment (Propensity) The fact that recent incidents (and some achievements) are predominantly swarms is empirical evidence for this. On the capability side, I don’t think a single agent could’ve hacked HuggingFace (at least currently), it probably took them many different agents. On the propensity side, I suspect a usual agent also would not (yet) have wanted to do this either. And yet, there is not yet a research agenda for multi-agent swarms. Previous multi-agent research has tended to focus on outside-of-lab scenarios (such as markets) and on achieving cooperation for the (presumably) post-AGI future. However, current multi-agent swarms are quite different from previous work: They happen within AI labs (rather than in the outside world). Furthermore, we don’t always want cooperation; instead, we usually want to stop collusion. Below, I give some of the research ideas I find compelling: Research ideas are scored roughly by how important for alignment they seem to me. This list isn't exhaustive, so I'd love to hear other ideas. I'll mark which ones we're pursuing at EuroSafeAI/Jinesis, so we can avoid duplicating effort (or join forces). Who we are: We are a part of EuroSafeAI, a new technical AI safety org focused on Multi-Agent swarms and Extreme Power Concentration. Located…