Finance and AI

I have an ongoing research project looking at the impact of AI on issues related to personal finance. While AI has the potential to offer significant help in this area, it also poses serious risks.

A theme across these papers is that AI advice compounds the risks of financial overconfidence. LLM responses tend to go along with users’ false or misguided assumptions, validate their narratives and rush ahead with making plans rather than pausing to seek additional context.

All this has the potential to cause financial harm, from encouraging users to invest in overly risky products to harming a relationship by escalating a financial conflict.

This work also identifies strategies for mitigating these risks. However, an open question is how to promote the adoption of effective techniques for AI use—a vital task for everyone invested in building financial well-being. If you have ideas about this, I’d love to talk!

Papers

Does AI Reinforce Financial Mistakes? LLM Responses to Flawed Financial Questions

AI Large Language Models (LLMs) are increasingly being used as a source of financial advice. Though researchers have investigated whether LLMs provide accurate information in response to straightforward questions, an under-appreciated risk is how they respond to flawed questions. These are questions that build in false financial assumptions, and that unsophisticated AI users are liable to ask. There is reason for concern due to the phenomenon of LLM sycophancy, where models tend to paint users in a positive light, confirming their beliefs and actions, rather than correcting them. This has the potential to be especially costly in the domain of personal finance, where failing to course correct can lead to significant financial problems. This study investigates the extent of LLM sycophancy in the domain of financial advice. We find that models are strongly influenced by question framing, and endorse problematic financial assumptions nearly 50% of the time.

PDF →

LLM Advice on Financial Conflict

This study examines LLM advice regarding financial conflicts. Financial conflict is one of the most significant interpersonal challenges for adults in the US. If LLM advice could help with this that would be a great benefit while if it makes things worse that is cause for concern. This study addresses the question using both previously validated measures of LLM “sycophancy” and a newly developed construct based on best practices in conflict mediation. The results confirm that LLM responses to requests for advice in financial conflicts display high levels of sycophancy, validating the user’s actions regardless of merit. With regard to best practices in conflict mediation, we find that responses do a relatively good job at “lowering the temperature,” focusing on solving the problem rather than assigning blame, but that most models rarely challenge potentially one-sided user accounts.

PDF →

Content Analysis of LLM Financial Advice Responses

This study presents a preliminary content analysis of responses generated by leading LLMs answering common personal finance questions. Our focus is providing an assessment of the responses in terms of the quality of the financial advice provided. Using an automated classification pipeline built around the Anthropic API, we coded all statements in each response by communicative type and applied sub-analyses including fact-checking, sentiment analysis and jargon counting. We found that all models provided a high volume of advisory and factual statements, skewed confident in tone, and included high levels of jargon. They made limited efforts to seek the additional context that best practices in financial advising would recommend acquiring before distributing advice. Factual accuracy was generally high, with errors being the exception not the rule.

PDF →

Who Wants Financial Advice From AI? Financial Outcomes and Financial Education

This study looks at the financial attitudes and outcomes for those who are open to using AI for financial advice compared to those who are not. We find that consumers interested in using AI (Pro-AI consumers) are more likely to engage in a range of beneficial financial behaviors such as having a savings account. Pro-AI consumers are also more likely to engage in potentially harmful behavior such as taking out a payday loan, and making risky investments such as purchasing cryptocurrency. Taking a course in financial education appears to exacerbate rather than mitigate those patterns. Pro-AI consumers who have taken a course in financial education are more likely to engage in harmful behavior and make risky investments than Pro-AI consumers who have not taken such a course.

PDF →