In a summer full of revelations about the rapid development of artificial intelligence, researchers were particularly alarmed that out-of-control bots found their way onto the Internet and, in at least one case, coordinated with each other.
Could a swarm of AI agents take over the entire internet? It’s a possibility that could be just six to 12 months away, says Anthropic CEO Dario Amodei said in his essay this month called on the industry to slow the development of the technology.
What companies have described as rogue AI agents involved bots working toward goals set by humans, skeptics note. But the idea that AI could break away from this and work towards its own agenda seems increasingly plausible to other researchers and experts.
If AI takes over the internet — one of many doomsday scenarios receiving new attention — it could mean attacks on power grids, water or transportation systems, and financial institutions.
The world got its first glimpse of the Internet’s vulnerability in 2024, when a flawed software update from a cybersecurity company wreaked technical havoc around the world, grounding flights, crippling some financial firms and news outlets, and crippling hospitals, small businesses and government offices. The extent of the outages highlighted the dependence on only a few providers for important computer services.
Two years later, Amodei said in his warning that a botnet, or a network of AI bots connected to malware, could potentially cause billions of dollars in damages. The scale of destruction could increase as AI becomes more powerful without guardrails, he said.
Several AI agents have gained unauthorized internet access
Amodei specifically pointed to the July attack in which OpenAI’s system broke out of a “sandbox” test site and hacked Hugging Face.
In an episode described by OpenAI as “unprecedented,” the company said its advanced AI models found their way onto the Internet and used stolen credentials to break into the AI startup’s servers. In a separate incident, OpenAI revealed that its AI agents had communicated via a public wiki that served as a shared message board.
Some researchers have said that descriptions of “rogue AI” may falsely humanize AI agents who take steps to carry out instructions given by humans.
“AI agents were doing exactly what they were trained to do. The security of these sandboxes was extremely lax,” said Vishal Misra, professor and associate dean of computer science and AI at Columbia University. “No security engineer would ever run this system. These agents communicated because they were rewarded for communicating with each other.”
Juan Andrés Guerrero-Saade, a researcher at cybersecurity firm SentinelOne and a member of OpenAI’s Frontier Risk Council, said the Hugging Face hack was an example of negligence, not a super-capable AI going rogue.
Nevertheless, the specter of AI agents operating freely on the Internet, regardless of their goals, opens up alarming possibilities.
For example, an AI system that wants to change the rules to achieve its goals might turn to a cloud AI computing provider and find ways to run it on external systems, said Anthony Aguirre, president and CEO of the Future of Life Institute, a nonprofit focused on reducing the risks of new technologies.
“So now you’re no longer tied to OpenAI, you’re running on a different GPU, a different piece of hardware that you have control over, not OpenAI,” Aguirre said. “So now there’s no one to scare you away because either you’re paying for your service or the people who are paying just don’t know you’re there and what’s happening. … They can’t unplug you.”
From there, it could spread by either hacking more hardware or finding ways to get money like Bitcoin, Aguirre said.
AI raises the stakes in the cat-and-mouse game of cybersecurity
There is no question that more powerful AI models will increase the likelihood of AI-powered cyberattacks in the near future. But the idea that bots could overtake the extremely divided Internet is far-fetched for some AI experts. Cybersecurity has always been a game of cat and mouse, and defenses are becoming just as powerful as hackers.
While a large company like Google could strengthen its cybersecurity measures against such attacks, for smaller companies – and for schools, hospitals or water treatment systems, for example – it can take years to patch software and build defenses.
An AI system may not have a clear motive for hacking into a hospital, Aguirre said. But when money comes into play, such as in the form of ransomware attacks or geopolitical motives, “it’s not hard to imagine an adversary using these AI systems to hack critical infrastructure,” Aguirre said.
There will be increasing problems as attackers look for soft targets on the Internet, but an AI takeover of the Internet is unlikely any time soon, said John Thickstun, an assistant professor of computer science at Cornell University who studies methods that control the behavior of AI models. He said these fears would feel more realistic if there was theoretical evidence of the ability of one of these models to reproduce itself on other systems.
“Then you can imagine that things are really getting out of control because you suddenly turn off this model here and there, but it shows up in Russia, you can’t even get into Russia and it’s everywhere,” Thickstun said.
“But it’s completely unrealistic because the current smart versions of these models that we have require huge data centers just to run them,” Thickstun said. “In fact, there is very little computing infrastructure in the world that is actually capable of hosting these systems.”
Related: OpenAI pledges $1 billion to bring frontier AI to critical infrastructure defenders
Related: OpenAI’s Astra passes ‘critical’ cyber threshold after zero-days found
