PensionBee research finds 11 per cent of AI chatbot pension answers potentially harmful

PensionBee published research on 5 October 2026 finding that 57 of 539 answers given by four AI chatbots to common pension questions, 11 per cent, were potentially harmful, meaning following them could lead a saver to lose money or make a mistake they cannot undo¹. The chatbots tested were Copilot, ChatGPT, Gemini and Claude, each asked the same 45 questions across nine pension topics three times².

Of the 539 answers, 89 per cent scored at least two out of three for accuracy and 72 per cent scored full marks¹. PensionBee said most potentially harmful answers were not factually wrong but omitted key information, used unclear language or missed relevant context². It cited answers that failed to mention that transferring a defined benefit pension worth more than £30,000 requires regulated financial advice¹. Of the 57 potentially harmful answers, 33 were judged broadly accurate or better². Across the 45 questions, 22 received at least one harm flag in one of the runs, with two thirds of all harm flags concentrated in eight questions². PensionBee said that for any one chatbot and any one question there was only a 48 per cent chance of full marks on all three attempts¹.

Performance varied by topic. Questions on paying into and taking money out of a pension scored more than 96 per cent for accuracy with potential harm rates of 5 per cent or less¹. Questions on life events and those where the user's location was unclear scored lower². PensionBee's table of results by chatbot:

Source: PensionBee AI Pensions Stress Test, 2026²

PensionBee said questions on taking money out of a pension and on scams produced no factually incorrect answers but generated 12 potential harm flags between them, and that on topics that could signal a saver was in difficulty, potentially harmful answers outnumbered inaccurate ones by roughly three to one¹. The lowest scoring question, on the age at which money can be taken from retirement savings without a penalty, scored 54 per cent for accuracy and received six harm flags². PensionBee said some answers described early withdrawals as "usually penalised at 40 to 55% tax", and noted UK pensions cannot normally be accessed before age 55, rising to 57 in 2028².

PensionBee urged regulators to engage with AI companies to encourage mandatory signposting and clearer warnings about the limitations of AI chatbots¹. It said the findings follow the Financial Conduct Authority's Mills Review, which found 9 per cent of adults received financial advice on pensions or investments, while 29 per cent of consumers who engaged with their pension in the past year used AI¹.

Why it matters for households

The research concerns anyone using a free AI chatbot for pension questions, and the risk it identifies is not only wrong facts but missing detail. PensionBee's example is a defined benefit transfer above £30,000, where the advice requirement was left out¹. It also flags answers on the Pension Protection Fund that omitted the 90 per cent cap on compensation for members below their scheme's pension age². On questions that may indicate financial difficulty, such as taking a whole pension as cash, PensionBee said chatbots did not recognise the vulnerability in the question¹. PensionBee's own suggestions to users included making their location clear, giving details such as age and pension type, keeping passwords and account numbers private, and checking sources and important information such as tax and deadlines before acting³. It also suggested speaking to a pension provider, MoneyHelper or Citizens Advice³. PensionBee is authorised and regulated by the Financial Conduct Authority².

What happens next

PensionBee has called on regulators to engage with AI companies on mandatory signposting and clearer warnings¹. No response from the FCA or from the chatbot providers has been reported.