|
|
|
Executives agreed that AI is delivering outsize value in some cases, but they found that in the majority of cases, it’s wasting more money than it’s worth. Often companies have a hard time distinguishing which is which. Better tools for observability are being developed but are still nascent, they said. |
|
David Hsu, CEO of AI harness company Retool, which works with customers across the Fortune 500, said he sees many companies getting a three-times to five-times return on their token spend, but that the results are incredibly uneven across uses. |
|
|
|
Retool CEO David Hsu spoke at the WSJLI Technology Council Summit about a simple AI-powered out-of-office responder that ran up a bill of about $10,000 a day. Photo: WSJ Leadership Institute |
|
“We believe that about 90% of the tokens you’re spending are probably negative ROI. It’s just that last 10% of tokens that are really driving a lot of that ROI,” he said. |
|
One money waster? An automatic out-of-office responder one employee at an unnamed company built with AI that turned out to cost $10,000 a day, Hsu said. |
|
It’s an extreme example but emblematic of a broader mismatch between spend and value that’s pervasive in today’s enterprise landscape. |
|
“I think the reality is, no, most people are not getting value from it [AI]. But those that are, are incredibly concentrated,” said Kate Smaje, senior partner and global leader of technology and AI at McKinsey and Co. |
|
According to Smaje, McKinsey’s latest research shows that 80% of workers feel they are getting a productivity boost from AI. However, only 6% of companies are getting real, measurable, financial ROI they can show to investors. |
|
“If I take 20 minutes on something and I multiply that out by my day, that doesn’t mean that somebody paid me 20% less or it doesn’t mean necessarily that I can get rid of 20% of people because it was all at the task level,” she said. |
|
So how can companies improve when it comes to AI ROI? |
|
|
|
Kate Smaje, McKinsey senior partner and global leader of Technology and AI Photo: WSJ Leadership Institute |
|
Glean’s Jain said the first step is investing in dashboards for extremely granular visibility into what everyone at the company is doing with AI. “When you allow people to build things, you give them immense power: the power to actually burn tokens and spend money.” |
|
He saw one employee in sales using AI to analyze lost sales opportunities, but it turned out the AI was costing more money than they would’ve gotten by signing that customer back. Better dashboards and observability platforms can help executives identify those use cases and cut back on them. |
|
It also turns out a lot of wastage comes from using generative AI or agents for things that they’re simply not needed for. |
|
For example, Hsu said he worked with one company that was proud of the fact that it had more AI agents than employees but was looking to optimize its costs. It became apparent, he added, that 95% of the problems the agents were solving were better suited to deterministic workflows—a shift that saved them around $20 to $30 million. |
|
Another key is making sure that when generative AI is the right play, companies aren’t necessarily tapping into the biggest frontier models for every use case. “You’re not trying to drive a Ferrari to the grocery store,” said Smaje. |
|
But one of the most critical points, she added, is workforce upskilling and organizational change management. |
|
When she looks at the 6% of companies that are seeing meaningful value from AI today, “they treat their AI transformation as a people transformation, not as a technology transformation.” |
|
She added, “What they’re trying to do is say, not ‘how do I bolt this onto existing processes and so on,’ but actually, ‘for every dollar I’m spending on technology, am I spending an equal and opposite dollar on change management, reskilling and upskilling.’ All of that humanware becomes disproportionately important.” |
|
|
|
The WSJ Technology Council Summit |
|
|
|
Indeed Chief Information and Chief Security Officer Anthony Moisant, second from left, spoke with WSJ Leadership Institute President Alan Murray, left, at the WSJ Technology Council Summit about why human judgment still matters. Photo: WSJ Leadership Institute |
|
Keeping humans in the loop. Amid a week of headlines about rogue AI agents and calls to slow AI development, panelists at the WSJ Technology Council Summit remained unshaken, convinced both that AI is transforming business and that real value from the effort depends on keeping humans in the loop. |
|
“I actually think human judgment is probably creating more lift in the business,” said Anthony Moisant, CIO and CSO of Indeed. |
|
They emphasized that the goal isn’t to replace workers, but to amplify them. “Some of the best AI deployments that we’ve done, the technology has been 20, 30% of the answer,” said McKinsey’s Smaje. “And then it’s what the humans have put on top.” |
|
That’s especially true in cybersecurity, where the stakes of getting it wrong are highest and the case for human oversight is clearest. “It is up to us, the humans are in charge of the AI. It is not the other way around,” said Galina Antova, CEO of AI cybersecurity company Kai. |
|
Panelists left no doubt that AI-powered transformation needs to happen, but argued it must be a culture shift, not just a tech upgrade. As Smaje put it, top-performing organizations “treat their AI transformation as a people transformation,” matching technology spend with investments in change management and upskilling. |
|
Said Marina F. Bellini, president, MGS & Digital Technologies, Mars, “we cannot think about the human side and the tech side separately.” |
|
More takeaways from Tuesday’s sessions: |
|
Open models lag behind frontier labs by only a few months, driving down the cost for both defenders and attackers alike.“In my estimate, the open-weight models are perhaps two or three months behind the frontier. And we also have to be very careful that we make sure the defenders always have the very best technology at their disposal.”— Oege de Moor, founder and CEO, XBOW |
|
Leadership teams are still facing intense board and market pressure to move rapidly. “There’s just a rational exuberance to a degree where… movement means progress, like its frenetic energy… The caution that I try to embed in the organization is like, look, we all know that we can run really, really, really fast in the wrong direction.” — Anthony Moisant, CIO and CSO, Indeed |
|
AI is helping break down long established silos between business leaders and IT teams. “We spent years training IT professionals to learn the business language, and now we are spending years training the business leaders to speak the technology language. So it’s good we’re all going to speak the same language finally.” — Marina F. Bellini, president, MGS & Digital Technologies, Mars |
|
|
|
Work-Bench co-founder and General Partner Jonathan Lehr . Photo: WSJ Leadership Institute |
|
Today the enterprise buy/build strategy is “buy to operate and build to differentiate.” “If I want to summarize what we’re seeing in the market right now in terms of buy versus build: It’s buy to operate and build to differentiate… Off the shelf software… can get you 80, 90% of the way there in terms of capabilities, but a lot of IT folks are actually spending time on that last mile.” — Jonathan Lehr, co-founder and general partner, Work-Bench
|
|
|
|
On Our Radar |
|
|
|
Guests look at a model of a large data center under construction in Abu Dhabi, giuseppe cacace/Agence France-Presse/Getty Images |
|
• Amazon Web Services data centers in the United Arab Emirates and Bahrain have been mostly offline since Iran launched drone swarms at them in the opening days of the U.S. war against the Islamic Republic. Now AWS is saying that it is
|