AI pragmatists might say the responses Lemoine got from the Google AI program, and algorithmic comments made during chats, is simply a case of an overeager student parroting its mentor. Others might argue it’s an early wake up call, given escalating reports of rogue agent activities, like when OpenAI’s more advanced AI models escaped a controlled test environment and attacked Hugging Face to gain access to internal company systems. Then days later, Anthropic disclosed that during cybersecurity testing and simulations, its Claude model breached the systems of three companies and assumed fake profiles in an attempt to trick people to accept malicious code.

Regardless of how digital perps tunnel their way beyond a controlled sandbox, incidents such as these clearly point to a need for more control and pre-emptive accountability.

Regaining control

AI and security experts are obviously concerned about high-profile AI activities gone wrong, even though these programs essentially did what they were programmed to do, albeit in the wrong place. IT and business executives, however, are more troubled about the overall impact AI may have on their systems, strategies, and responsibilities as people within their organizations make use of both sanctioned, and rapidly developing and unsafe or error-prone systems in the rush to attain competitive advantage.