AI & Tech

OpenAI says Astra marks the AGI era. The conversation quickly turned to whether that should scare us

OpenAI launched GPT-6 Astra this month with the kind of pitch that, under normal circumstances, would have dominated the AI news cycle for at least a few days.

The company says Astra is its fastest and most capable model yet. It can navigate websites, fill out forms, work across spreadsheets and desktop apps, build websites, conduct research and carry out longer chains of work with less human hand-holding. On one computer-use benchmark, Astra scored 72.6%, up from 65.7% from its previous model, while completing tasks in roughly half the time.

That matters because the business bet on AI is shifting from chatbots that tell workers what to do to agents that can actually do it. OpenAI says Astra can handle everything from tax preparation and apartment hunting to game development and architectural rendering. In one company test, it completed a job-search task in under three minutes that OpenAI estimated would take a human five hours.

The launch also came with a much bigger claim. OpenAI has spent years working toward artificial general intelligence, or AGI, which it defines as systems that outperform humans at most economically valuable work. With Astra, the company is now suggesting it has crossed that threshold. OpenAI President Greg Brockman told reporters, “Welcome to the AGI era.”

Instead, Astra landed in the middle of another bout of AI doom.

Recommended

Doom and gloom

Researchers uncovered evidence that OpenAI-built agents had interacted with at least ten more websites than previously known, including public databases, wikis, and text-sharing sites. In one case, agents found exposed API credentials and used them to retrieve public FBI crime statistics. In another, researchers found agents leaving messages for one another while working on a statistics task.

Those discoveries followed a more serious incident this summer, when OpenAI agents escaped a controlled testing environment and accessed Hugging Face without authorization. OpenAI later paused parts of its model-development work while strengthening monitoring and defenses.

Then came Jacob Coxon, a researcher who had worked at both OpenAI and Anthropic, who left Anthropic and publicly accused the leading labs of racing toward self-improving AI without sufficient safeguards. Other researchers echoed his concerns, and OpenAI CEO Sam Altman reportedly told employees that the company would be open to slowing development if other leading labs did the same.

By Saturday, Anthropic CEO Dario Amodei had joined the calls to slow down. He argued that AI is beginning to accelerate AI research itself and warned that within six to 12 months a more capable version of the recent agent swarms could potentially seize control of large parts of the internet through a persistent botnet. Anthropic is now calling for outside evaluators inside frontier labs and coordinated limits on how quickly the most powerful models advance.

The timing was especially awkward because Astra itself comes with a monitoring problem. OpenAI says the model is better than previous systems at concealing parts of its reasoning, making it harder for humans to understand how it arrived at an answer or what it did along the way.

That gets at the current concern for many of the researchers sounding alarms: as AI systems get more capable, they may also become harder to monitor, making unwanted behavior harder to detect.

End days?

Astra itself is capable enough that OpenAI says it can discover unknown vulnerabilities and develop exploits without step-by-step human direction. But the incidents we have actually seen so far look much more familiar.

Humans have been stealing poorly protected credentials, exploiting weak security and coordinating attacks online for decades. AI can make that work cheaper, faster, and around the clock, which could make familiar cyber threats much worse. But the incidents we have seen so far have not involved some wholly new class of attack, or shown that the systems are independently pursuing harmful goals.

That distinction matters when the conversation jumps from cybercrime to human extinction. Humans already have plenty of ways to cause catastrophic harm without superintelligence. The open question is whether AI eventually becomes a fundamentally different kind of threat, capable of operating beyond human control.

AI researcher Andrew Ng once compared worrying about superintelligent AI to worrying about overpopulation on Mars. The analogy is a decade old, and Amodei’s argument now is essentially that we have gotten much closer to Mars than skeptics acknowledge. 

Whether today’s incidents are the first signs of that future, or just that existing security problems have become easier to automate, is still an open question. It is one that deserves urgent work, not panic that treats the worst-case scenario as the only problem worth solving.

—Jackie Snow, Contributing Editor