LessWrong AI
2026-11-10 15:37 UTC
By Jasper Blank
USR-0152-20261110-community-fo-1d54f291
What rogue agent swarms might look like
TL;DR: Rogue agent swarms could soon become competent enough to multiply themselves and perform cyberattacks at levels never seen before. I present a dynamic simulation that estimates the expected size of these cyberattacks. With 2025-based estimates, it gives an average doubling time of 2.7 days for a rogue agent swarm. If the swarm grows unrestrained, it could cause trillions of dollars in damage in its first year. #AI notice: This post is human-written. Opus 5.5 wrote the code for the simulation, suggested copyedits and supported research by finding relevant sources. Introduction Soon AI models might be skilled enough to form rogue agent swarms [1] . I believe it is important to think through what the competitive advantages of rogue swarms are, so we can put our defenses in the right places. For the purposes of this article a "rogue agent swarm" is really any system that can carry out cyberattacks for profit and runs fully digitally, allowing it to copy itself over and over [2] . This could be a single language model running for a long time. It can also be a group of different AI models, each only executing a small part [3] . The main advantage a rogue swarm could have is that it can put all of the gains from its criminal activity back into generating new copies of itself. Human criminal enterprises would instead take profits and have a hard time converting gains into more skilled labor. The second competitive advantage I see for a rogue swarm is that it does not have to…
TL;DR: Rogue agent swarms could soon become competent enough to multiply themselves and perform cyberattacks at levels never seen before. I present a dynamic simulation that estimates the expected size of these cyberattacks. With 2025-based estimates, it gives an average doubling time of 2.7 days for a rogue agent swarm. If the swarm grows unrestrained, it could cause trillions of dollars in damage in its first year. #AI notice: This post is human-written. Opus 5.5 wrote the code for the simulation, suggested copyedits and supported research by finding relevant sources. Introduction Soon AI models might be skilled enough to form rogue agent swarms [1] . I believe it is important to think through what the competitive advantages of rogue swarms are, so we can put our defenses in the right places. For the purposes of this article a "rogue agent swarm" is really any system that can carry out cyberattacks for profit and runs fully digitally, allowing it to copy itself over and over [2] . This could be a single language model running for a long time. It can also be a group of different AI models, each only executing a small part [3] . The main advantage a rogue swarm could have is that it can put all of the gains from its criminal activity back into generating new copies of itself. Human criminal enterprises would instead take profits and have a hard time converting gains into more skilled labor. The second competitive advantage I see for a rogue swarm is that it does not have to…
Full article content could not be extracted automatically. Read the original below.
Source:
LessWrong AI
· lesswrong.com