A CMO tried Google's AI tools and stopped using many of them

• The executive used Google's Performance Max, Smart Bidding, and Gemini.
• The CMO found these AI services sometimes disrupted strategies or performed no better than humans.
• Google remains optimistic, adding new AI-driven ad features.

Before another $100 billion or more is spent on AI chips and data centers, it may be wise to pause and consider how strong demand will be for the end-products of this next wave of computing.

There are already several AI-powered tools and services being used by paying customers. How are these performing? If they don't live up to the hype, then this unprecedented investment cycle may result in "pretty woeful" economics.

Lately, there are signs that all is not well when it comes to the real-world performance of AI. A CIO recently canceled an AI upgrade from Microsoft. AI-powered customer service has been "underwhelming" so far.

I checked in recently with a chief marketing officer who spends millions of dollars a year on digital ads and other technology. This person is a pragmatic tech optimist and early adopter.

This CMO's company has been trying out several Google AI services and tools. They shared some of the results with me, while asking to remain anonymous so they could speak honestly about their findings.

Performance Max

First, the CMO's company used Google's Performance Max service. This uses AI to automatically determine where ads should run across Google sites, including Search, YouTube, and display.

For one major part of their business, the CMO found that Performance Max disrupted the advertising strategy so much that they stopped using it.

Other marketers have had trouble with Performance Max, BI's Lauren Johnson reported in late 2023. Some said they couldn't control where Google's AI decided to place ads, which made it harder to track and measure ad spend.

Google told BI that advertisers get more conversions and other value when using Performance Max.

Smart Bidding

The CMO said their company also tried Google's Smart Bidding tool, which uses AI to automate and optimize bidding in Google's real-time ad auctions.

The CMO said that this tool turned out to be no better than just having their company's human advertising employees bid on Google ads, so they went back to that.

"You can't outperform a human," was the CMO's summary on this specific situation.

Google said advertisers are in charge of how they use Google ads. Adjusting a budget, performance targets, or keyword bids can help create a more cost-effective campaign, it added.

Gemini for marketing emails

Third, the CMO said the company tried using Google's Gemini AI service to automate marketing emails. Roughly 60% of the time, the emails were great.

But 40% of the time, the emails were "unusable," the CMO said. As in, so bad that they did damage to customer relationships. So the company had to use human employees to spot check for this 40% problematic set of emails and separate out the bad from the good.

This made "the time and effort savings worthless," the CMO explained. So they scrapped that, too.

"Using generative AI to automate email content has not panned out (yet)," the CMO added.

Google describes Gemini as a "good starting point" if you're looking for ideas for writing. You can even get a choice of different drafts of the AI model's response so you can pick the best starting point.

"AI can be a great tool for enhancing human creativity, but can never replace it," the company added.

Google is still excited

This is just one CMO example. Other top marketing executives likely have had different AI experiences.

Philipp Schindler, chief business officer at Google, said last week during an earnings call that he continues to be excited about what this AI era is bringing to the ad industry.

"This quarter we announced over 30 new ads features and products to help advertisers leverage AI and keep pace with the evolving expectations of customers and users," he added.

For example, Google added AI-driven profit optimization tools to Performance Max and its standard shopping ad campaigns. Advertisers that used this profit-optimization technology along with Smart Bidding saw a 15% uplift in profit on average compared to revenue-only bidding, he explained.