Can ChatGPT generate superior SOAs?

On a recent episode of the ifa podcast, founder and director of Neilson & Co Wealth Management, Ben Neilson, explained how he is using ChatGPT to help provide financial advice.

“We used [ChatGPT] to reduce the need to create content initially. I said there's four or five pages here that I don't care about, and I don't want to bill to do,” Mr Neilson said.

“If you look at the content that comes out with the super contribution recommendation, it's the same. It's always the same. It's just the amount and the person are different. I'm sitting in the office thinking, ‘Why am I billing for this? How can I get this done?’ So, we used ChatGPT to write recommendations, and then I thought there's probably more to that.”

He went a step further and decided to evaluate the compliance of four of the largest Australian licensees with regulations.

“We loaded 28 SOAs (statements of advice) from four of the largest Australian licensees and I said ‘can you create a tool to evaluate these’ against the Corporations Act, the regulatory guides, the code of conduct,” Mr Neilson said.

“It came out with this thing called psycholinguistic model, which is basically a language accuracy tool. And it basically came back and said, ‘The four largest licensees’ SOAs are actually quite low in levels of persuasiveness and accuracy and narration’. And ChatGPT’s was incredibly high,” Mr Neilson said.

“Generally speaking it costs between $800 and $1,000 to get one super recommendation and takes between three and three and a half hours to complete. If you look at our office, generally speaking, it's about $600 and about two hours. If you look at ChatGPT, it takes about 28 minutes and costs about $200.”

But the AI is not infallible and Mr Neilson said there were specific areas that the model struggled with, such as section 961B of the Corporations Act, which spells out that a provider must act in the best interests of the client.

Ultimately, Mr Neilson said that advisers shouldn’t be worried about AI replacing them, adding that it will free them up to do the important work.

“A lot of the feedback that we got was suggesting that advisers thought it would replace them. But there's actually a subsection in the Corporations Act, which is 921Y, which basically says that you have to be a registered provider and ChatGPT is unregistered,” he said.

“We're doing the parts that the clients value, we're doing it cost effectively and the net result is significantly better, but it takes us less time. This is a recurring theme through all of my research. This is how the thesis addresses barriers to accessing financial advice. We do the parts that matter, and we outsource the parts that don't.

“Historically advisers have just done everything and then wondered why no one wants to pay them $20,000 or $6,000 for it because there's no value there.”

To hear more from Mr Neilson, click here.