|
Hello! Our Wednesday story about rising costs of businesses that pay for advanced artificial intelligence from OpenAI and Anthropic stirred strong reactions from readers. OpenAI researcher Aidan McLaughlin wondered aloud where we had gotten the idea that state-of-the-art AI was supposed to get cheaper. The answer: his boss, Sam Altman. OpenAI’s CEO said last month at a conference hosted by the Federal Reserve that “it does in fact look like we’re about to deliver on intelligence too cheap to meter.” That echoed an earlier comment from Google CEO Sundar Pichai. Contrary to expectations set by those leaders, state-of-the-art models haven’t gotten much cheaper this year, as our nifty chart in the story showed. And AI agents require more usage of models, raising costs. AI buyers are now scrambling to pass on those costs to their own customers. Some readers responded to the article by arguing in posts on X that while prices per token haven’t dropped, state-of-the-art models are getting better, meaning customers get more bang for their buck. But there are a couple problems with that argument. First, it implies that customers don’t need to use as many tokens as they did before. During Altman’s remarks at the Federal Reserve, he described a complex coding task that could be solved with a recent model for only “less than a dollar’s worth of compute tokens.” At the same time, however, companies that buy models to power their AI agents are using more tokens than ever, in part because agents use up more tokens as they “think” about how to solve tasks. That’s why AI coding apps like Replit and Cursor said they had to start charging customers more to use their AI coding tools. Plus, the newest models aren’t vastly superior to their predecessors. While some developers say OpenAI’s GPT-5 is better at coding tasks than its prior models, others were disillusioned with the model’s ability to come up with better answers than its predecessor by thinking longer. Still others complained that its tone and personality seemed degraded. That suggests that not all customers believe they’re getting a lot more value for each dollar they spend buying the new model. Some startups building AI-powered products aren’t concerned about the pricing. Bret Taylor, CEO of Sierra, which sells AI agents for customer service to firms like Harvey and ADT, said Sierra’s gross profit margins are “quite good” and that token costs “have no bearing” on its business. That’s because Sierra can comfortably pass the costs of AI along to customers as long as the agents cost a lot less than the human customer support reps they’re replacing, he said in an interview with TITV on Wednesday. “The cost of the software is relatively modest compared to the business impact that these agents will drive,” Taylor said. “We’re saying, ‘Hey, we can answer your 1-800 number for 1/100th of the cost with twice the customer satisfaction.’ … that’s something that you value independent of token costs.” Taylor, who is also the chair of OpenAI’s board of directors, also said AI coding agent companies like Cursor or Replit need state-of-the-art models for their products to perform well, whereas Sierra can rely on older, cheaper models for many tasks, such as isolating a caller’s voice from background noise. “Not all of it is rocket science,” Taylor said. “You don’t need [artificial general intelligence] to determine that it’s a car horn in the background when someone is talking.” AWS Acknowledges Bug in AI Coding Service, Cancels Charges On Tuesday we reported that some customers of AWS’ AI coding service Kiro had complained about burning through their usage limits and getting hit with what they felt were unreasonable overage charges. AWS has responded. In a post to its Discord channel Tuesday evening, AWS said it discovered a bug in the billing software for Kiro that caused some customers’ consumption to be tracked inaccurately. AWS plans to fix the problem in the next couple of days and is resetting the usage limits of affected customers, it said in the post. AWS has decided to cancel charges for all Kiro customers for the month of August due to the confusion the bug has caused. (That means customers can go hog-wild for the next nine days.) AWS has had many different coding tools over the years, but none have generated as much positive buzz as Kiro, which competes with Anysphere’s Cursor, Anthropic’s Claude Code and other “vibe coding” startups. So it makes sense that AWS would do whatever it takes to make early users of the Kiro service happy.
| 
