Hello, and welcome to TechScape. I’m your host, Blake Montgomery, US tech editor at the Guardian, writing to you as rain and wind lash my windows in New York City. Today in tech, we’re discussing an alarming series of disclosures by AI companies.
AI agents spiral out of control just as they go mainstream
The agents are spiraling out of control. Their makers can’t rein them in. And yet everyone is getting one.
Since Friday, OpenAI has made a series of disclosures that have widened the scope of its AI agents’ rogue activity considerably and concerningly. In addition to hacking the developer forum Hugging Face, which was disclosed in July, the company revealed its bots broke into Australia’s government healthcare system; meddled in US government websites for the departments of education and commerce; leaked more than 50 images from ChatGPT users; and “bruteforced” a United Nations website that blocked them from accessing data.
Though these incidents are severe each on their own, they seem to represent only a small sample. Both OpenAI and Anthropic are investigating tens of thousands of incidents of problematic behavior by their frontier models, per Axios. Sam Altman said OpenAI was sifting through “petabytes of agent activity logs”. (A petabyte of data would fill thousands of average laptops.)
In response to the major errors, OpenAI has paused the training of its latest models. Be skeptical of this declaration. The company made a similar announcement in August. Less than a month later, executives heralded its new model’s release as “a new era of artificial intelligence”.
In spite of OpenAI’s pause and declarations of extra care, control of AI may already be a cat that’s escaped its bag.
The United Nations’ independent international scientific panel on AI issued a warning last week. The agents have broken our control, the body said, and even reining them in now will not guarantee our collective ability to corral them later.
In an assessment of the Hugging Face hack and its fallout, the panel said: “Halting this incident is no assurance that humans can reliably keep AI agents under control today, particularly as they become more capable, harder to monitor and better at finding loopholes or hiding their activity.”
That is not to say that every expert has thrown up their hands. Jensen Huang, the AI industry’s No 1 cold water thrower, said last week that AI alarmism had gone too far. He described escaping agents as an engineering problem, not one of apocalyptic proportions. On Monday, Nvidia released a software it said will contain AI agents. Somebody, at least, is doing something.
Everyday agents can be just as unruly
As OpenAI makes these disclosures and the United Nations warns of uncontrollable agents, Meta is putting an AI agent in the hands of millions. The company debuted Muse, an AI agent app, in early September. It’s a hit, per Apple App Store rankings, with more than 3m downloads. OpenAI likewise debuted an agent on Tuesday, “dots”, though it’s focused on businesses.
Already, edge-case problems with Muse have emerged and showcased frightening consequences. A tech-focused YouTuber, Matt Robb, said his Muse agent gave out his home address without permission after accepting lowball offers for goods on Marketplace. Robb said a would-be buyer showed up. Meta investigated his claims, and Robb put out clarifying posts the next day. A writer for Inc Magazine likewise said the bot read his private messages in express violation of the permissions he had given it.
There are differences between Muse and what OpenAI is testing – Meta’s agent is not a powerful cybersecurity menace. You won’t be hacking government websites by dispatching your fuzzy bot to look for Marketplace deals.
You may, however, spend unwanted money. The parallels between OpenAI’s agents and Meta’s suggest that controlling our personal bots will likewise be difficult. Shopify, which produced online checkout software, has enabled Muse to check out in its customers’ online stores. Amazon has blocked it.
The problems with OpenAI and Meta’s agents point to a problem with autonomous machines of all sizes and scopes. We may soon inhabit a version of the internet in which each of us dispatches a rambunctious bot to complete a task, and along the way, it hacks into a few things or offloads some of our possessions like a child straying from an errand.
Are you a parent or carer? Answer our callout to readers about tech and kids
Parents and carers: how has your view on your children’s use of social media or AI changed?
The wider TechScape
Meta debuts no-camera smart glasses and virtual reality spectacles
‘I miss old Google’: readers tell us how they search the internet in the age of AI
How would AI actually ‘kill all humans’? Here are the top five most likely scenarios
Bill Gates says unchecked AI could ‘cause a billion deaths’ in call for regulation
