Don’t ask ChatGPT for financial product recommendations

an article written by Ron Shevlin, Chief Research Officer at Cornerstone Advisors, Senior Contributor at Forbes, Fintech Snark Tank (bit.ly/FintechSnarkTank)

A new survey from the Motley Fool reveals some surprising—and, frankly, hard to believe—statistics about Americans’ use of the Generative AI tool ChatGPT for financial advice.

An email from Pollfish with the subject line “54% of Americans rely on ChatGPT for financial product recommendations” caught my attention. So I clicked through to the study which claimed:

54% of Americans have used ChatGPT for finance recommendations. Six in 10 Gen Zers and Millennials, half of Gen Xers, and a third of Baby Boomers said they’ve received recommendations for at least one of eight financial products. Credit cards and checking accounts—cited by 26% and 23% of respondents, respectively—were the products most frequently asked about.

Half of consumers said they would use ChatGPT to get a recommendation. That said, few expressed in getting a recommendation in for most products. For example, 25% said they’d want a recommendation from ChatGPT for a credit card—and the percentages go down from there.

Respondents were “somewhat satisfied” with ChatGPT’s recommendations. On a 5-point scale (1=not satisfied, 5=very satisfied), the average overall satisfaction rating was 3.7, ranging from 3.6 from Gen Zers and Baby Boomers to 3.8 from Millennials and 3.9 from Gen Xers.

A variety of factors regarding the use of ChatGPT for financial product recommendations are important to consumers. The most important factors determining the use ChatGPT to find financial products are: 1) the performance and accuracy of the recommendations; 2) the ability to understand logic behind the recommendations; and 3) the ability to verify information the recommendation is based on.

“Have Used” Does Not Mean “Rely”

Shame on Pollfish for poor interpretation of the data. Nowhere in the results of the study did respondents ever say that they “rely” on ChatGPT for financial product recommendations. They were asked if they “have used” the tool—not if they rely on it.

The data itself—from a The Motley Fool survey—is misleading and doesn’t address some important questions about how consumers are using the #generativeai tool to get recommendations about financial products.

(…)

Should Consumers Use ChatGPT For Financial Advice?

While ChatGPT may be able to help with stock picking, consumers should not rely—let alone “use”—the tool for financial product recommendations. Why?

ChatGPT doesn’t have the latest product information. Rates, fees, and product performance change on a frequent basis in banking. ChatGPT, however, is only trained on data through the end of 2021. Any product recommendation the tool makes is probably being made using out-of-date data. Consumers said that “the ability to verify the information a recommendation is based on” is an important factor governing their decision to use ChatGPT for financial product recommendations, but there appears to be no actual “information” used in the recommendations.

Good recommendations are based on actual use and preferences. How can anyone—or ChatGPT—make a product recommendation without understanding the customer’s historical use of a product and his or her preferences? It seems unlikely to me that most consumers who used ChatGPT for financial product recommendations adequately supplied their product preferences. And they’d have been crazy to have given ChatGPT their transaction history or actual financial information.

The article in full here