The GenAI jitters: Is there enough demand for $1 trillion in AI spending?

When you build a lot of new AI data centers, it's very hard to undo that if end-demand turns out to be weaker than expected.

• There's a debate raging over returns on Big Tech's heavy AI investments.
• Google's CEO said he'd rather the company over-invest right now.
• Amazon CEO Andy Jassy said over-investing can result in "pretty woeful" economics.

Business Insider chief tech correspondent Ashley Stewart wrote a story this week based on what seemed like a pretty innocuous topic.

It was about a CIO at a pharmaceutical company who had employees try Microsoft's 365 Copilot AI-powered productivity software. After 6 months, he cancelled because the new features weren't worth the extra money. You can read the article here.

The piece really grabbed BI subscribers' attention, and I've been thinking about why.

I've realized this anecdote goes to the heart of the generative AI boom — and whether it can continue.

Big tech companies are forecast to spend $1 trillion on data centers, real estate, chips and other gear to build AI models, tools and products. That is only going to pay off if there's actual demand for these offerings from real users, especially corporate customers with money to spend.

The CIO story is not a good sign of strong demand. As Ashley wrote, if a pharma company can't spend an extra $180,000 on these AI tools, that's a problem for the entire tech industry.

Big tech executives were peppered with questions on this broad topic during earnings calls this week.

"There's an industry debate raging around the capex requirements around generative AI and whether the monetization is actually going to match with that," Keith Weiss, a tech analyst at Morgan Stanley, said during the Microsoft call.

Some big tech companies have already whittled away a large chunk of their cash reserves chasing this AI trend. Weiss and the rest of Wall Street wants to know the return on this spending.

For more than a year, the assumption has been that generative AI will stoke a massive wave of new demand. What if that demand turns out to be weaker than expected?

The perils of over-investing

The problem is that building lots of expensive data centers is a long-term commitment. If you end up not needing all that infrastructure, you can't just make it magically disappear. Those are very large costs that are mostly fixed.

Amazon CEO Andy Jassy put it well on his company's earnings call.

"If you end up actually with too little capacity, then you have service disruptions, which really nobody does because it means companies can't scale their applications. So most companies deliver more capacity than they need," he said. "However, if you actually deliver too much capacity, the economics are pretty woeful and you don't like the returns of the operating income."

When Google CEO Sundar Pichai was asked a similar question, he said Google will choose over-investing, rather than miss out on potential AI revenue opportunities.

"The risk of under-investing is dramatically greater than the risk of over-investing for us here, even in scenarios where if it turns out that we are over-investing," he explained.

Google is a $2 trillion corporation, with oodles of data at its fingertips. I suspect that investors want the company to get this right, rather than over-invest and end up with woeful economics.

Nadella and 365 Copilot

Microsoft CEO Satya Nadella was probably the most sanguine of all. He said the software giant is seeing strong demand signals from customers.

A major piece of evidence he shared was about demand for Microsoft's 365 Copilot service. This is the same product that BI's Ashley Stewart wrote about earlier this week.

"When I think about what's happening with M365 Copilot as perhaps the best Office 365 or M365 suite we have had," Nadella said, while adding that customers are "coming back buying more seats."

This is exactly the opposite of that dissatisfied pharma CIO. Hopefully, for the whole tech industry, this turns out to be an outlier not a trend.