|
On Saturday, September 12, Dario Amodei, the chief executive officer of the artificial intelligence company Anthropic, published a 3,800-word essay calling for AI companies to slow down their improvement of AI models.
Amodei expressed concern that AI models are themselves pushing advances faster than engineers can understand them. He noted that July’s OpenAI–Hugging Face incident, in which programs designed to hack into systems found weaknesses that permitted them to escape the “sandbox” in which designers were testing them for about a week before anyone noticed had, luckily, been relatively harmless, but warned that “in my opinion, a swarm that possessed greater capabilities but a similar level of misalignment could have caused catastrophic damage.” Within a year, he warned, such a swarm could take over the entire internet, causing hundreds of billions of dollars in damage.
In his essay, Amodei called for AI companies to commit to giving embedded third-party evaluators access to their work, rather like the regulatory supervisors in banking. He called for AI companies in democratic countries to establish safety standards and limit their rate of progress. And he called for “[t]he US and other democratic governments [to] attempt to coordinate with authoritarian governments, to the extent this is possible, while taking seriously the challenges of verifying compliance.”
Today on Face the Nation, Amodei explained to host Jo Ling Kent: “My view here is it has always been very strange that this technology is being built by a private company. People ask me that question all the time. Why isn’t this being built by government? And the strangest thing about it is, I agree with them. I’m uncomfortable. Government didn’t build this technology. This company, this technology came from the private sector, and we—Anthropic, and I would hope other companies, have done everything we can to try to have legitimate oversight mechanisms.
“I think the government and the public need to have a stake. And this is why we’ve supported regulation of the technology. Regulation constrains the private companies. Regulation allows the public and its elected representatives to have a say, and limits what the private companies can do.”
Asked if he would be willing to give the technology itself to the government, Amodei answered that he might be willing to give it to the right combination of governments. “I want to be very clear about this,” he said. “I am concerned that one single government could abuse this technology just as easily as a single company could. But I think a combination of democratically elected governments—I don’t know about hand over, but some kind of oversight, some kind of joint governance. Again, that would be the work of years, but I wonder if that’s the direction we need to go in.”
As Mike Isaac of the New York Times noted, Amodei’s call came days after an Anthropic researcher, Jacob Coxon, resigned, posting on social media that neither OpenAI nor Anthropic was “acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives.” They are, he said, building “superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources.”
“The people building AI earnestly believe that it could kill us all by the end of the decade,” Coxon said. He claimed that people at OpenAI might not yet realize what they’re constructing, while those at Anthropic do understand but believe “they are locked in a race to get there first—they believe no one else will act responsibly, so they must do it themselves, despite the risk.”
Critics are skeptical, suggesting that AI leaders are hyping their product to make it appear more valuable than it really is. There is widespread concern that the massive spending on AI infrastructure like data centers and chips might not deliver the revenue and profits that would justify such spending. In June, Kate Brennan, associate director of independent research institute AI Now, told Aimee Picchi of CBS News: “The returns are not coming in, and the claims that are being made, in terms of efficiency or productivity numbers, are not netting out.”
Others wonder if Amodei’s public concerns aren’t designed to get the government to regulate AI in such a way that it creates a framework that would make it hard for smaller companies to break into the market.
Still, Kate Conger of the New York Times noted last week that in July, more than 1,300 employees from Anthropic, OpenAI, Meta, and Google’s DeepMind—all leading AI companies—signed an open letter asking the U.S. government to regulate AI to slow down its development. Conger also notes that in August, more than 100 tech companies offered their models to hospitals and infrastructure systems to enable them to guard against cyberattacks powered by AI.
After Amodei’s essay appeared, Sam Altman, the chief executive officer of OpenAI; Elon Musk, who has been increasing spending on AI through his SpaceX rocket company; and Demis Hassabis, chair of Google DeepMind, all posted their support for slowing down the pace of AI improvements.
On October 30, 2023, President Joe Biden issued Executive Order 14110, calling for the “Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence.” The document said AI must be safe and secure. It called for the promotion of “responsible innovation, competition, and collaboration” and specified that AI must engage U.S. workers and advance equity and civil rights as well as privacy and civil liberties. The development of AI must protect consumers, it said, and the government must make sure the technology is deployed responsibly.
Revoking this executive order was one of the first things Trump did on January 20, 2025. Published in the official register on January 23, Trump’s order “Removing Barriers to American Leadership in Artificial Intelligence” called for reviewing “all policies, directives, regulations, orders, and other actions taken” under Biden’s order and suspending, revising, or rescinding them. “[W]e must,” the order said, “develop AI systems that are free from ideological bias or engineered social agendas.”
During his second term, Trump and his sons have invested heavily in companies tied to the AI boom. Trump has called AI data centers “the oil of the next 50 years” and says they are delivering wealth and investment to communities in the U.S. In July he insisted that data centers are “Cash Cows,” creating taxes and jobs that “amount to LIQUID GOLD!”
But the American people disagree. A YouGov poll from late August showed that only 24% of Americans think the construction of data centers is a good thing, while twice that number, 47%, say it’s bad. A majority of Americans, 61%, don’t want one in their town. In response to their growing unpopularity, the administration is seeking to exempt data centers from having to notify the public about how much air pollution they will release.
As Cat Zakrzewski, Violet Jira, and Nitasha Tiku of the Washington Post reported, when asked today about the calls to slow down the development of AI models, Trump dismissed them, saying that the U.S. needed to stay ahead of China. He brushed off the warnings calling for the slower pace coming from the industry’s leaders.
“I think you have a lot of negative forces that are bringing it up that shouldn’t be bringing it up,” Trump told reporters. “And they’re bringing up things that won’t happen.”
David Sacks, the venture capitalist who heads the President’s Council of Advisors on Science and Technology, accused the AI leaders of trying to avoid “massive product-liablity exposure if your products enable a truly damaging cyberattack.”
A Wall Street Journal article today by Richard Rubin and Justin Lahart offered a different perspective on the fight over regulating AI. In a piece about the growing U.S. debt, they note that the Trump administration insists it can overcome the rising deficits it’s mounting and the debt that has recently hit 100% of the nation’s gross domestic product and topped $40 trillion through growth.
The idea that the U.S. could sustain high spending with low taxes by growing its way out of debt has been a driving force in the Republican Party since the 1980s, but as Rubin and Lahart note, the U.S. has not had the sustained 3% growth such a scenario requires since the 1990s, when companies were adopting computers and baby boomers were at their peak employment. During those years, under President Bill Clinton, the U.S. wiped out its deficit.
But then President George W. Bush pushed through another big tax cut and launched two unfunded wars, and both deficits and debt climbed again. By this century, the authors note, the conditions of the 1990s were reversed: baby boomers are retiring and productivity is slowing.
And yet Treasury Secretary Scott Bessent told an audience at Southern Methodist University last week that he expects to see 3% growth again after the end of the war on Iran. “[T]he underlying economy is very, very strong,” he said, “and I think reaccelerating.” This expected growth seems to be what’s behind the administration’s faith that it can continue to cut taxes while dramatically increasing spending on the military and the Department of Homeland security.
Last week, Rubin and Lahart note, Trump told the Fox News Channel: “We’re gonna take care of the 40 trillion over a period of time through growth. We’re growing at a faster rate than we’ve ever grown before.” While the U.S. is not, in fact, growing at a record pace—growth during Trump’s second term has sat at 1.9%—it appears the administration may be looking at a giant boost in productivity led by AI as its Hail Mary pass.
—
Notes:
https://darioamodei.com/post/we-must-pace-the-frontier
https://www.nytimes.com/2026/09/12/technology/anthropic-dario-amodei-ai-slowdown.html
https://www.nytimes.com/2026/09/09/technology/anthropic-researchers-raise-alarm.html
https://www.cbsnews.com/news/ai-bubble-tech-selloff-investment-consumer-business-demand/