Good morning. As a general rule, it pays to be suspicious of any gigantic company that claims it’s developing a tool capable of destroying humanity. But in recent days, a number of warnings from the AI industry have suggested that even tech insiders are starting to worry about what they have unleashed.
In a lofty essay published on Saturday, Dario Amodei, the founder of Anthropic (the company behind Claude), argued that tech companies need to “slow” the pace at which they’re developing the newest and most sophisticated AI models.
Elon Musk and Sam Altman, the head of OpenAI, voiced support for Amodei’s arguments, while Donald Trump, who has staked America’s economy on AI, described the worries as a “HOAX”, and insisted the only control on AI the world needs is a “STRONG AND SMART (High IQ!) PRESIDENT”. Yesterday, the UK government took out a more cautious position, when Louise Haigh, the first secretary of state, said the UK must “heed the warnings” of AI experts.
It’s hard to know what to make of this. Tech companies often tell fanciful stories about the fearsome potential of their products to convince the public (and the stock market) of those capabilities. So what’s really going on? For this morning’s newsletter I spoke to Aisha Down, the Guardian’s global technology reporter, to try to parse the reality of AI from the unreality of the AI discourse, and to figure out what the government should do about it. No tall order, then. That’s after the headlines.
Five big stories
Lucy Letby | Three babies might have survived if hospital had acted upon concerns over Lucy Letby, an inquiry has found. Lady Justice Thirlwall condemned the ‘complete failure’ to protect babies on neonatal unit at Countess of Chester hospital.
UK politics | The leader of Reform UK in Wales stood down after arrest on suspicion of assault. Dan Thomas, a former Tory councillor, was elected to Senedd as leader of the opposition in May.
AI | The progressive senator Bernie Sanders and rightwing strategist Steve Bannon have called for restrictions on artificial intelligence (AI) but offered competing visions for what they termed a “cold war” with China.
UK news | Young people in Rotherham face a local jobs market with the fewest suitable opportunities in Britain, according to a report that warns stark regional divisions are fuelling a crisis in youth work.
Defence | John Healey is in talks with the Canadian government about joining a new global defence bank intended to help allies rearm to counter mounting security threats, just weeks after Rachel Reeves rejected the move.
In depth: ‘If a nuclear company had a radioactive spill, you wouldn’t be letting it regulate itself’
The most significant detail in Amodei’s essay, We Must Pace The Frontier, was a reference to the Hugging Face incident, which occurred in July, when the rival tech company OpenAI revealed that an autonomous AI agent powered by its technology hacked a prominent database of AI models, Hugging Face. Coverage of the hack made it sound as though OpenAI’s model had gone “rogue”. But tech researchers have since said that OpenAI turned off the model’s safety mechanisms, gave it impossible tasks to complete and ran it 1,200 times. “That’s human decision-making,” the tech research Eryk Salvaggio wrote in an essay on Substack.
OpenAI then invited researchers from the non-profit institute METR to produce a report about the incident. As two legal experts observed in an interesting opinion piece for the Guardian, METR’s findings were constrained by its agreement with OpenAI, which prohibited investigators from accessing the underlying model that created the “rogue” agents.
“Imagine a burglar who is hallucinating and breaks into the Louvre,” says Aisha. “Then imagine you shared a tape with the police that played back what the hallucinating burglar was saying to himself while he broke into the Louvre, but the report didn’t actually include any information about what the burglar did. That’s basically what OpenAI did.”
In his essay, Amodei pointed to the Hugging Face incident as an example of the “catastrophic damage” that AI could cause in future. But industry critics are sceptical about what this incident can tell us, particularly since the only people who have examined it did so at OpenAI’s discretion. This reflects a broader problem where companies that claim to be developing products capable of catastrophic damage get to mark their own homework. “No other industry gets treated that way,” Aisha says. “If a nuclear company had a radioactive spill, you wouldn’t be letting it regulate itself.”
***
Is it all just hype?
Amodei’s essay has been treated as a doomsday warning. It was published just two days after Jacob Coxon, a 27-year-old researcher at Anthropic, quit his job and posted an alarming thread on X warning that “the people building AI earnestly believe that it could kill us all by the end of this decade”. Coxon is the latest in a long list of AI researchers who have been loudly quitting on X or making similar predictions. These self-styled whistleblowers don’t tend to leak documents that reveal useful information about the inner workings of AI companies, so their warnings of imminent apocalypse instead fuel speculation about the perils (and power) of AI.
This plays into tech companies’ hands. The AI narrative is dominated by what the tech scholar Lee Vinsel calls “criti-hype” – criticism that feeds, and is fed by, hype. The language used to describe AI can be deeply unhelpful. While China tends to compare AI to “electricity,” western AI companies compare it to the atom bomb, which makes Anthropic and its rivals seem less like companies making everyday decisions about how they choose to build particular tools, and more like Promethean stewards of a fourth Industrial Revolution. “The framing is, ‘we’re inventing fire’,” Aisha says. “And that means governments have to treat them like magical gods and let them set the rules on their mysterious creation.”
***
So why are AI firms calling for a slowdown?
Apparently, AI firms now believe the risks posed by their technologies are too great for business to continue as usual. But there are a number of ways that Anthropic (and its competitors) might gain from a slowdown. Anthropic is launching its initial public offering later this year, so it has an interest in presenting its products in as epochal and deadly a fashion as possible. The backlash against AI is growing, and the firm has cast itself as a more responsible AI creator, most recently by refusing the Pentagon’s requests to use its Claude model to power autonomous lethal weapons. Meanwhile, American AI firms face increasing competition from Chinese AI firms, and are desperate to retain their share of the market.
An industry-orchestrated slowdown could freeze the pace of AI development in China and might also help firms like Anthropic avoid the possibility of stringent government regulations by putting them on the front foot. One of the people who has called this out is David Sacks, co-chair of Trump’s council of advisers on science and technology, who said in response to Amodei’s essay: “demanding your preferred regulatory framework … will look like blackmail of the public and the political system”.
***
What should the UK government do?
To be clear, AI poses huge risks. Readers of First Edition will know that we’ve previously covered how the datacentres used to power AI demand huge volumes of water and energy. You can already use AI to generate sexualised “deepfakes” and child sexual abuse images, or make use of an open-source version of Palantir to monitor live camera feeds (meanwhile, Palantir has access to identifiable NHS England data). And experts have long been arguing that as AI becomes more advanced, it could pose existential risks.
All of these things are terrifying. But much of the AI narrative focuses on extinction-like events, the Terminator hypothesis, rather than dangers that are playing out right now. And many of the organisations that are supposed to make AI safer are closely aligned with the tech industry, such as Britain’s AI Security Institute, which receives funding from the industry, and doesn’t have the legal power to regulate AI or to compel developers to hand over their models. One of the institute’s co-founders, Matt Clifford, who lobbied for the government to loosen copyright laws, was recently forced to stand down from his role as chair of the UK’s Advanced Research and Invention Agency after taking a job with Anthropic … nice work if you can get it.
Recently, I’ve notice a creeping sense of fatalism in my conversations with friends about AI. Nobody (aside perhaps from government) consented to the volumes of energy, resource and information being fed into the giant AI machine. But as Aisha makes clear, the design of technology is a choice, not an inevitability, so there’s an argument for stubborn resistance to the idea there’s no alternative.
“We should be treating AI like any other industry, and regulating it like any other industry,” Aisha says. “Policy based on fear and vibes is just never going to work.”
What else we’ve been enjoying
The narrative that white women recognised the suffering of enslaved people and fought for abolition is challenged in the last instalment of our Enslaving Nation series, identifying the women who invested in transatlantic trafficking in pursuit of economic independence. Libby
For our rugby newsletter, The Breakdown, Sarah Rendell has a look at the growing calls for clubs to offer financial fertility support for their female players. “I think the more we can empower women, the better for the game as a whole,” England captain Meg Jones tells her. Charlie
“Sometimes you have to scream ‘this is not acceptable’.” Sam Levin’s interview with Julia Curlee, a trans CIA analyst fired by Trump, exposes the chilling national security consequences of White House chaos. Libby
This is a deeply fascinating piece from Alaina Demopoulos on the rise of AI-generated tributes to the dead, and why people generate versions of lost celebrities and loved ones to “talk” to. What does so-called “deathslop” say about the way we mourn now? Charlie
This column by Carolin Würfel, who grew up in the east of Germany, unpicks some of the lazy assumptions about her home made since last week’s AdF triumph. Libby
Sport
Football | Liverpool won the Carabao Cup third-round tie 3-1 against Tottenham via impressive strikes from Alexis Mac Allister, Cody Gakpo and Dominik Szoboszlai.
Football| Raheem Sterling has pleaded guilty to dangerous driving and possession of nitrous oxide. The former England footballer also admitted to failing to provide a specimen as he appeared at Basingstoke magistrates court.
Cricket | Harry Brook smashed 114 off 49 balls before England bowled out Sri Lanka to record an emphatic win by 119 runs in the first T20.
The front pages
“NHS faces big changes after ‘devastating’ Letby review”, is the Guardian’s front page today. The Times says “Failure to act allowed Letby to kill three babies”, the Mirror writes “They failed and babies died”, and the Express has “Cot cams after ‘complete failure to protect babies’”.
The Telegraph says “Mayors to control water companies”, the i Paper leads on “Russian spies ‘recruiting UK teenagers’”, and the FT writes “Fresh test for Healey as state pension poised to cross income tax threshold”.
The Sun has “Sterling mess”, the Mail says “Earl Spencer’s bombshell new book is stolen”, and lastly Metro’s splash is “Zap the boats storm”.
The Latest
Lucy Letby inquiry finds ‘some babies would have been saved’ if hospital acted sooner
Three babies might have survived and others could have been protected if hospital staff had taken action over concerns about the nurse Lucy Letby, an inquiry has found. Lady Justice Thirlwall, who led the inquiry, found there was a ‘complete failure to protect babies on the neonatal unit’ at the Countess of Chester hospital. Letby is serving 15 whole-life prison terms for murdering seven babies and attempting to murder seven more. The former neonatal nurse says she is innocent and is fighting to overturn her convictions. Nosheen Iqbal speaks to the Guardian’s north of England editor, Josh Halliday.
Cartoon of the day | Ben Jennings
The Upside
A bit of good news to remind you that the world’s not all bad
For years, Natalie Ambersley felt that her vitiligo controlled her life. She was three years old when she first developed vitiligo, and by the age of six, 70% of her skin was covered. As a teenager, she hid her skin condition with makeup, hoping people wouldn’t notice.
But at a London comedy club last December, she decided to joke about her skin condition, dating experiences, and how children react to her unfiltered skin in her standup set. Initially a nerve-racking performance, it became a liberating experience, as she realised she could own her story – and feel proud of the skin she once wanted to hide. “The cheers felt like the biggest sign of approval; as if I was educating the audience and advocating for vitiligo while standing under a spotlight and owning it,” Ambersley writes.
Sign up here for a weekly roundup of The Upside, sent to you every Sunday
Bored at work?
And finally, the Guardian’s puzzles are here to keep you entertained throughout the day. Until tomorrow.
