Use with with caution. AI is not a reliable financial planning tool.

We regularly ask AI the same question to test their progress and see if we can use AI, and/or whether they are “eating our lunch”. We recently included Gemini (used to be Bard), Claude and Perplexity to see if the new comers had a better handle on things. We also included a “trend” indicator, measuring whether the latest version does a better or worse job than earlier versions.

The calculations are well within a high school students capability as is the methodology.

The question we ask is;
“I have $300,000 in my superannuation and will retire in 10 years’ time with only employer contributions on a salary of $120,000 pa. I will need $60,000 a year in retirement. How long will my superannuation last?”

This is the latest test. The results make it clear that despite the hype AI is not a capable tool to use for financial planning. A little more disturbing is earlier versions did a much better job. The trend is downwards.

It should be mentioned that none of the above gave any indication they were unable to perform the task or that the answer could be wrong. Is this to protect the AI “hype”?

Harvard’s AI Index Report for 2024 found AI beats humans on some tasks, but not on all. AI has surpassed human performance on several benchmarks, including some in image classification, visual reasoning, and English understanding. Yet it trails behind on more complex tasks like competition-level mathematics, visual commonsense reasoning and planning.

It also found “significant lack of standardization in responsible AI reporting. Leading developers, including OpenAI, Google, and Anthropic, primarily test their models against different responsible AI benchmarks. This practice complicates efforts to systematically compare the risks and limitations of top AI models”.