Why is it so hard to get ROI from AI? Because building from first principles isn’t easy

At Fortune Brainstorm Tech, executives said that finding value from AI starts with strategy and continues with process reinvention.

Hello and welcome to Eye on AI. In this edition…highlights from Fortune Brainstorm Tech…Anthropic walks back a controversial decision around its new Fable model…OpenAI considers slashing prices…Meta looks to boost subscription revenue…and Anthropic’s Dario Amodei has a new AI policy manifesto.

It’s Jeremy here. First, I want to thank Sharon Goldman for her work as my co-writer on this newsletter over the past two years. Sharon, who normally writes the Thursday send of Eye on AI, is going off to become her own one woman media empire. We wish her all the best in her new venture.

I spent the first part of this week at Fortune Brainstorm Tech in Aspen, Colorado, where AI dominated every conversation, on stage and off. On Wednesday morning, I even hosted Fortune’s first Eye on AI breakfast, where it was great to meet some readers in the flesh. We discussed the reasons many companies are struggling to realize return on investment from AI.

Principles before profit

Manoj Bohra, the CTO at asset management firm State Street, said that part of the problem was the amount of what he called “foundation work” that companies needed to do to enable AI projects to succeed, especially in regulated industries. Having the right data in the right places with the right governance controls was a key first step. Next, he said, comes thinking through and mapping work flows and processes before attempting to automate any part of them. All of this work takes time and investment and it was wrong for companies to expect a return on that investment in just one or two years, Bohra said. He compared it to building a bridge. “No one judges the return on investment of a railroad bridge in just a single year,” he said. 

Bill Briggs, the chief technology officer at Deloitte, said that a lot of businesses had failed to do the hard first principles work of thinking through what they were trying to achieve strategically with AI. Instead, they often rushed to launch any AI use case at scale in order to seem AI-savvy, even though sometimes those use cases didn’t move the needle on firm-wide revenues or profits. He also said that many firms simply dropped AI into existing processes rather than rethinking workflows from the ground up in an AI-native way. The result, he said, was that existing inefficiencies “get weaponized at scale” by AI agents. He said those who drew parallels to the early age of industrial electrification, where businesses simply replaced steam engines with electric turbines and found productivity gains elusive, were correct. Most firms are trying to automate existing processes rather than doing the harder work of redesigning them. But only by re-engineering processes will businesses see a big productivity boost. 

A first principles approach is also critical for Kathy Pham, head of AI at ReviveHealth. She said we often optimize for the wrong thing. For instance, the question of whether parents should allow AI to read bedtime stories to their children varies depending on how parents view the purpose of story time. If some see it as simply a means to getting a child to sleep, then it might be fine to have AI read a bedtime story. But if the purpose is actually about the parent spending focused, intentional time with their child, then it would be defeating to allow AI to read the story. Businesses often have processes that have, over time, become divorced from their intended purpose and dropping AI into these processes can often fail to deliver value, she said. 

Stephen Balaban, the cofounder and CTO of AI infrastructure firm Lambda, told the breakfast that he didn’t think AI was actually ready for many use cases outside of software development and that it was probably a mistake to push AI agents into other parts of a large company. But he also noted that until six months ago, AI agents also weren’t capable of autonomous software development. Now they are. And he said companies should start preparing now for the moment in the coming year or two when AI models would be capable enough to power agents in other domains. He also said that businesses were rightly demanding that services firms start charging for outcomes rather than for the amount of headcount they use to staff a project.

Wen Sang, the cofounder and chief operating officer at Genspark, an AI unicorn that markets systems of AI agents to specific professional verticals, said that enterprises should look for easy wins that can drive revenue. He used the example of advertising firms that have shifted from hiring artists to produce static storyboards for pitch meetings to instead using AI to create video prototypes of a television ad campaign. The result is an increased chance of winning business at less cost than it took to produce the old story boards.

Faraz Shafiq, the chief AI product officer at Wells Fargo, said the bank had tried to build fundamental horizontal “building blocks” that work across different lines of business. This includes a unified AI agent platform and AI governance infrastructure. But then within each business line, the bank has looked to reinvent processes end-to-end with help from the bank’s domain experts. The question then becomes, he says, how you value and measure some of the returns from the productivity gains AI produces? In some cases, that’s easy, he said. For instance, the bank has seen a 25% increase in new account openings thanks to the use of AI tools. But if a banker can spend more time with a customer, what is the value of that human relationship? Sometimes it isn’t easy to measure in just immediate new business. Instead, the goodwill created by the banker being able to offer more personalized service might lead to more revenue for the bank over years or decades.

Overseeing AI agents ups the need for human connection

Beyond the Eye on AI breakfast, I had the privilege of interviewing Anthropic’s Head of Claude Code, Boris Cherny, during a mainstage fire side chat on Monday. Among the many interesting things Cherny said was that in a world where engineers are spending all of their time just supervising AI agents (and Cherny himself says he often has hundreds of AI agents running tasks in parallel), getting his team together in-person, to build trust and esprit de corps and to allow for mentorship, was more important than ever.

In a mainstage interview with Hyatt CEO Mark Hoplamazian and Snowflake CEO Sridhar Ramaswamy, both executives said they thought that software companies that have begun to try to restrict third party AI agents from accessing their platforms would ultimately lose. Ramaswamy said it was a sign of weakness that some of his rivals were even thinking of doing this, while Hoplamazian said customers wouldn’t tolerate this kind of “toll gating.”

Meanwhile, Mistral cofounder and CTO Timothee Lacroix told attendees that “AI sovereignty” was about countries seizing control of the parts of the stack that they could. Mistral, which has benefitted from positioning itself as a European alternative to American-headquartered AI labs, is best known for its AI models but has recently begun building its own data center capacity too. But it has no plans, for the moment, to build its own AI chips, Lacroix said. Both he and Defined.ai CEO Daniela Braga said that data was a key piece of the sovereign AI puzzle, with countries increasingly looking to ensure that there was a public benefit to any AI model built on public data and also trying to ensure that culturally-sensitive data sets, such those that contain the stories and language of indigenous peoples.

I also had a great discussion with Adaption Labs’ CEO Sara Hooker and SambaNova CEO Rodrigo Liang. While the two disagreed on the extent to which massive LLMs will continue to be the future of AI, both agreed that there is a desperate need to optimize the efficiency of AI workloads. Hooker wants to do this partly at the model level, by exploring new AI architectures that are inherently more efficient, and partly by building much more adaptive harnesses and systems around AI models. Liang wants to do it by routing AI inference loads to the best chips for any particular query. He thinks data centers will increasingly need to be heterodox, with a mixture of GPUs, CPUs, and specialized AI inference chips from providers like SambaNova.

You can check out all of Fortune’s coverage of Brainstorm Tech here. And with that, here’s more AI news.

Jeremy Kahn
jeremy.kahn@fortune.com
@jeremyakahn

This story was originally featured on Fortune.com