ChatGPT is better at predicting how stocks will react to news headlines than traditional models, new study shows

A new study suggests that ChatGPT is very good at gauging news sentiment and predicting how stocks will react.

• Florida researchers asked ChatGPT to analyze the sentiment of news headlines to forecast resulting stock moves.
• They said their study "demonstrates the value of ChatGPT in predicting stock market returns."
• But ChatGPT still has flaws, and the researchers advise investors to not rely on it for investment decisions.

ChatGPT can't see the future, but it already has value for investors looking to predict future moves in the stock market. 

That's according to a new research paper published Monday in the Social Science Research Network by two University of Florida finance professors. 

The researchers fed the chatbot over 50,000 news headlines about companies that date back to October of 2021 and asked it to determine whether the headlines were good, bad, or irrelevant news for the companies' stock prices. They used this sentiment analysis to compute a numerical "ChatGPT score" and analyzed whether these scores were predictive of the company's stock market performance the next day. 

The researchers found a statistically significant positive correlation between these scores and the next-day stock performance for the companies they analyzed. Firms with higher scores tended to see better returns than those with lower scores.

The study also found that ChatGPT outperformed other "traditional sentiment analysis methods" that also use data from headlines and social media to forecast stock movements — although the researchers admit they did not test every single one of these methods in this study. 

"In short, our study demonstrates the value of ChatGPT in predicting stock market returns," the researchers wrote.

They added: "Our results suggest that incorporating advanced language models into the investment decision-making process can yield more accurate predictions and enhance the performance of quantitative trading strategies." 

When the researchers compared the performance of ChatGPT with traditional methods, they found that those other models didn't add any predictive power over their ChatGPT-derived sentiment scores.  

"Once you use ChatGPT, using the other sentiment measure is not helpful for predicting," Alejandro Lopez-Lira, a co-author of the paper, told Insider. 

—Alejandro Lopez Lira (@alejandroll10) April 10, 2023

Investors should still refrain from relying solely on ChatGPT

Lopez-Lira said that in the future, AI tools like ChatGPT could be used to improve the stock market's efficiency by incorporating news faster into stock prices. He also said these tools could replace some investment analysts. 

Ever since ChatGPT was rolled out last November, users have tested its limits. They've asked it what stocks to invest in, used it to earn money through side hustles, and even asked it to help start a business. As much as the chatbot has amazed, many of its limitations have surfaced as well. 

Lopez-Lira said that investors should "exercise caution and not rely solely on ChatGPT or similar AI models," in part because there are several areas where the AI needs to improve when it comes to stock market prediction. 

First, ChatGPT isn't "connected to the internet" and therefore can't access the latest information available. 

"ChatGPT does not have access to any recent data beyond its training cutoff in September 2021," he said. "This limitation means that the AI model may not know more recent market trends, news, or developments that could significantly impact stock prices and investment decisions."

ChatGPT also currently struggles at processing large texts and "number crunching."

"It cannot process large amounts of numerical data, such as firms' accounting data," he said.

Addressing these concerns, Lopez-Lira said, could "vastly increase the prediction capabilities" of the chatbot. 

"As the field of AI-driven finance continues to expand, the insights gleaned from this research can help guide the development of more accurate, efficient, and responsible models that enhance the performance of financial decision-making processes," the researchers wrote in the paper's concluding line.