|
On Friday, Aaron and Kevin’s article about Salesforce calling out the ingrained limitations of large language models struck a major nerve, as have our other in-depth articles about Salesforce AI challenges this year. Business people saw the piece as further evidence that marketing of generative AI got ahead of what the technology could actually do. Software engineers largely responded to us (and on X) with some version of “duh, how is this news?” We assured these engineers that the article had a lot of news value, and our internal analytics bear that out! To those of us who report on AI, though, the piece and reactions to it added to our confusion about this moment. Dueling narratives are everywhere. On the bright side:
- Nvidia is an Energizer bunny of chip growth and is raking in most of the profits from the AI field. Other providers of AI-related hardware are doing great too.
- OpenAI and Anthropic continue raking in revenue—but not profits—from chatbot subscriptions to individuals (and some businesses) and sales from their application programming interfaces to customers such as Microsoft’s GitHub Copilot and coding app Cursor.
- Cloud providers such as Microsoft, Amazon and Google are growing faster than they used to, in part due to server spending by such AI developers on Nvidia-powered servers.
- Around 20 young “AI native” startups such as Cursor and Cognition are generating more than $100 million in annualized revenue, according to the Generative AI Database.
- LLMs are crushing complex performance benchmarks
- Scientists say LLMs are aiding their work in energy research and other fields, and there are kernels of evidence that this technology can solve novel math proofs, for instance.
On the other hand:
- Public market investors no longer take much stock in promises by cloud providers like Oracle that say they’ll collect hundreds of billions of dollars of revenue from OpenAI. That’s because it’s nearly impossible to know whether OpenAI can get anywhere near the $200 billion in annual revenue it projected in 2030 from the more than $13 billion it will generate this year. The past is no guarantee of the future, and that kind of projected growth is unprecedented.
- Most chatbot users don’t know how to use the products effectively, according to OpenAI employees. (Which also means OpenAI still has a chance to show them how it’s done!) And top Microsoft executives privately say they are concerned that employees at many companies are paying for Copilot AI features that they’re not using.
- LLM performance benchmarks seem to have little correlation with real-world performance.
- Luminaries such as Ilya Sutskever and Andrej Karpathy, both co-founders of OpenAI, have been pretty down on LLMs’ ability to generalize, or handle a wide variety of tasks, and are critical of the effectiveness of methods of improving their performance, while other researchers say the AI field needs an overhaul.
- Timelines for AGI are being pushed out, and other automation predictions—such as Dario Amodei’s aggressive one related to the percentage of code produced by AI—didn’t pan out.
- 2025 was not the year of AI agents after all, despite what Salesforce, Microsoft and OpenAI (and many others) had led people to believe. (Remember OpenAI’s plan to release $20,000-a-month PhD-level AI agents?) AI agents powered by LLMs have struggled to catch on in a big way at corporations, despite numerous experiments.
- Getting AI apps to work properly inside of corporations often requires a lot of hand-holding and other support from the AI providers.
In the middle of all of this, the biggest technology firms keep investing in, and doing more business with, the biggest AI developers and upstart cloud providers—and vice versa. In just the past month, Microsoft bet big on Anthropic, Anthropic bet big on Google’s AI chips, and Amazon is betting big on OpenAI. And Nvidia has been spraying its cash everywhere to keep the wheels turning going. That merry go-round understandably raises questions about how we’re supposed to separate organic growth from these kinds of megadeals between a handful of tech providers, partners and customers. ‘Normal Technology’ It’s tempting to fall back on Arvind Narayanan, Sayash Kapoor and Tim O’Reilly’s suggestion that genAI is just another piece of software—a “normal technology,” as they put it—that will take a long time to diffuse throughout the economy due to a variety of real-world factors that limit how quickly it is adopted. Yes, LLMs aren’t just magic that turn your data into digital workers that use Excel or handle customer support the way humans do. But there are a lot of people—including Satya Nadella himself, as we just reported in detail—now working on making LLMs and agents useful in enterprises. So today’s agents are probably the worst they’ll ever be. And AI revenue is rising, even at Salesforce, so something must be working. We’ve covered case studies of success outside of coding this year, such as Novo Nordisk’s use of Anthropic models to write first drafts of text for regulatory filings and IT services giant Kyndryl cutting its cybersecurity incident response team by half with the aid of AI from Palo Alto Networks. We covered plenty of AI failures and challenges too. Skill Issue? Some engineers who responded to Friday’s Salesforce piece said the problem might stem from the way Salesforce and its customers are able to handle—or not, as the case may be—the latest AI technologies. In other words, the engineers said, it’s a skill issue. There’s some truth to that. Beyond software firms, AI success stories often happen in industries that have always been first to embrace new software—finance, insurance and pharmaceuticals in particular. For the rest, it will take time. The same is true for individual workers, whose personal and professional productivity can be meaningfully boosted by good ol’ ChatGPT. They just don’t seem to know it yet. If you're wondering what this means for asset values (public and private technology stock prices) heading into 2026, I won’t try to be an oracle. All I know is I’ve never seen a more sensitive stock market. When Aaron earlier this month wrote about Microsoft lowering salespeople’s growth targets tied to quotas for newer Azure AI products, Microsoft’s stock fell 2%, which was strange because the piece was about a tiny fraction of its business, not overall AI-related sales. And when data center financier Blue Owl walked away from the chance to fund another Oracle facility for OpenAI, Oracle’s stock fell 5%. Our colleague Anissa on Monday explained why that made little sense to her. In other words, the market is hyper-reactive and driven by headlines and narratives, not necessarily by financial fundamentals. Small signals are being blown out of proportion. How sustainable is that? As for the application of AI in the real world, it’s plain to see we’re at the dawn of a new era. It just isn’t an overnight revolution for most businesses. There’s a lot to look forward to, as a bevy of startups pave the way for new categories of LLM products, such as in health. (We encourage you to subscribe to AI Agenda to learn about their revenue growth.) And products involving AI-generated video, imagery and audio are already upending fields such as video games, marketing and design. Progress is never a straight line, but it’s happening.
|
