AI will be a lot less profitable than previous tech waves

• Big tech is spending $1 trillion on generative AI.
• Generative AI won't scale as well as old-school software businesses.
• RBC analysts see GenAI leaving software profit margins 'structurally' lower.

Amazon, Microsoft, and Google spent more than $48 billion in the second quarter, mostly on data centers, according to Synergy Research Group.

Big tech companies are forecast to spend an eye-watering $1 trillion on this stuff. A lot of the capex is fueled by expectations that generative AI will be the next big tech thing after cloud computing.

The natural question that follows is this: What will the returns be on these massive AI investments?

Analysts at RBC Capital Markets came up with an early answer this week, and it's not great.

"Long-term software gross margins will structurally be lower as a result of GenAI," they wrote in a research note.

When software went from "on-premise," where companies ran it on their own computers, to the cloud, where it runs on remote rented machines, gross profit margins fell from 90% to 75%, according to RBC.

The shift from cloud computing to generative AI will bring software margins down further, to roughly 60%, the analysts estimated.

Gross margin is a simple measure of profitability that takes revenue and subtracts the cost of goods sold.

In the software business, gross profit margins have traditionally been in the 90% range. That sounds like a lot, but this is why this sector is so attractive to investors, and why software companies have such high valuations.

It costs a lot upfront to develop new software. But once it's created, the cost of making new versions and distributing those to customers is almost zero. So every time you sell more software, your profits get bigger and bigger.

When tech investors bang on constantly about "scale," this is what they mean. The software business traditionally has massive scale: More sales = way more profit.

Software scale in the GenAI era

Why might the software business be less profitable in the coming AI era?

"It may be difficult to drive so much efficiency on the P&L with GenAI," the RBC analysts wrote, referring to companies' profit and loss statements.

Generative AI is expensive to develop, but also to run.

There's AI model training. That involves buying incredibly expensive GPUs from Nvidia. Then putting those AI chips into servers that need special cooling and networking inside huge data centers. These facilities use massive amounts of electricity, which also costs a lot and requires pricy upgrades.

This doesn't even include the cost of the data needed for AI model training. Big tech companies and startups are trying to avoid paying for most of this. But collecting and cleaning this data is still expensive.

Once the AI models have been trained, they have to be run. This is the inference step, where the models are shown new data or requests and they infer useful things from the information. This step requires pricey chips, too, and is more of an on-going expense.

This is not like the old on-premise software business, where each new sale was almost 100% profit. Every time an AI customer uses a GenAI service, there are a host of costs borne by the provider.

For example, ChatGPT costs OpenAI $700,000 a day to operate, industry analyst Dylan Patel estimated last year.

Revenue upside

The RBC analysts weren't all gloom and doom, though.

They expect GenAI to be so revolutionary that customers spend a lot more on new AI-powered software. That should increase software revenue by 2x or even 3x current levels, they estimated.

With the software market so much bigger, there may also be more "profit dollars" available, even if profit margins are lower, the analysts also explained.

Profit dollars are what executives and analysts fall back on when profit margins are slipping. It's a measure of the absolute profit a company generates.

For example, if a company has $100 million in revenue and 10% profit margins, that's $10 million in absolute dollar profit.

If this theoretical company sees revenue jump to $300 million, but the margin slips to 8%, that's still $24 million in income — more profit than before.

"While we expect GenAI to pressure margins, we believe long-term gross profit dollars… will be higher in a post-GenAI world," the RBC analysts concluded.

The big assumption here is that GenAI stokes huge revenue increases. I hope RBC Capital Markets is right, otherwise these huge AI investments may produce "pretty woeful" economics.