What happens when AI gives wrong advice to your customers

Two people at a desk reviewing a printed document, one pointing to a specific section
TL;DR

When AI tools give wrong, misleading, or biased advice, UK businesses bear the legal consequences, not the AI vendor. UK law has no mechanism to claim against a model. Many AI vendor contracts disclaim accuracy, and the ICO confirms businesses are data controllers accountable for their AI outputs. The risk bites hardest when AI output reaches customers without review, affects real people, or is mistaken for professional advice.

Key takeaways

- AI hallucination rates in business contexts run between 3% and 27%, meaning systematic human checks are essential for client-facing output, not occasional spot-checks. - UK law has no mechanism to claim against an AI tool. When AI gives wrong advice under your brand, the legal and regulatory exposure sits with your business. - A Canadian tribunal ruled in 2023 that Air Canada could not avoid liability by arguing its chatbot was a separate entity from the company. The business owned the outcome. - The sharpest risk sits at three points: customer-facing chatbots answering substantive queries, AI-drafted documents in regulated sectors, and AI used in hiring or screening decisions. - Three practical first steps: a one-page internal AI policy naming what AI can and cannot draft for clients, a named sign-off rule for AI content before it reaches customers, and a check with your insurer about professional indemnity cover for AI-related errors.

A UK recruitment agency added a chatbot to handle candidate questions in late 2023. Basic stuff: application steps, pay scales, working arrangements. A candidate relied on what the chatbot said about a salary band, took the role on that basis, and raised a claim when the actual offer came in significantly lower. The AI vendor’s terms ran to 34 pages and said outputs were not guaranteed accurate. The firm’s professional indemnity policy had a carve-out for AI-assisted communications. The settlement came out of the owner’s pocket.

That scenario is composite, but every element comes from patterns documented in UK legal commentary and published case law. The AI gave wrong information. The firm paid.

What counts as bad AI advice?

Bad AI advice is any output that is wrong, incomplete, or misleading on something a person actually relies on. The problem for service business owners is that it rarely reads like nonsense. AI tools produce confident, professional-sounding text even when the underlying facts are wrong. Vodafone’s SME research puts AI hallucination rates between 3% and 27% depending on the system, a wide band for any client-facing deployment.

In practice, the patterns cluster around a few types. Wrong factual information: an AI tool drafts a client email stating incorrect HMRC filing deadlines, and the email goes out unchecked. Misleading explanations: AI-generated proposal text oversimplifies a regulatory requirement in a way that sounds right but skips a legal step. Biased outputs: a hiring tool scores candidates in ways that disadvantage women or minority groups because of what the training data reflected. Advice that encourages non-compliance: an AI-drafted privacy policy that skips mandatory UK GDPR disclosures. The common thread is that the content looks authoritative enough to be passed on without a second check.

UK law does not recognise AI tools as legal persons. There is no mechanism to claim against a chatbot. When AI advice causes harm under your brand, any legal or regulatory action is directed at your business. Many AI vendor contracts disclaim accuracy, and the ICO confirms that the business deploying AI is the data controller, accountable for outputs even when using a third-party provider.

The Air Canada case is the clearest illustration in published case law. In 2023, a Canadian tribunal found the airline responsible for misleading information its chatbot gave a passenger about bereavement fares. Air Canada argued the chatbot was a separate entity. The tribunal rejected the argument and ordered compensation. Press coverage ran internationally for months after the original mistake.

UK legal commentary consistently confirms the same position. The business owner is responsible for what a chatbot says on their website, regardless of who built the model underneath it. If the firm is FCA-regulated, the duty runs further. The FCA’s Consumer Duty requires firms to ensure AI outputs are fair, clear, and not misleading, and to avoid foreseeable customer harm. Choosing a third-party AI tool does not delegate that duty away.

Where does it actually show up in service firms?

The three spots where bad AI advice causes real problems in service businesses are: customer-facing chatbots answering queries about products, prices, or processes; AI-drafted proposals and client documents where a staff member reviewed the format rather than the substance; and AI-assisted decisions about candidates or customers, such as hiring tools or credit-scoring that reflect biases from the training data.

The chatbot risk is well-documented. Vodafone’s research found that 50% of consumers feel frustrated by chatbot interactions, and nearly 40% describe their experiences as negative overall. Consumer frustration is manageable. The deeper problem is when a customer acts on wrong chatbot information: a wrong cancellation policy, a deadline that no longer applies, a safety procedure that was out of date. That is the moment the claim arrives.

The proposal risk is less visible. AI-drafted documents move through approval workflows quickly because they look right. The issue arises when the content is wrong in ways that require sector-specific knowledge to catch. A small consultancy drafting a proposal for a client in a regulated sector, with AI filling the compliance sections and no one with regulatory expertise reviewing it, is an exposure waiting to happen.

Hiring is where the legal teeth are sharpest. AI tools used in recruitment can embed historic biases and produce discriminatory outcomes that breach the Equality Act 2010, even when the employer had no discriminatory intent. The ICO and the Equality and Human Rights Commission have jointly warned about exactly this pattern.

When does the risk actually bite, and when can you relax?

The risk is highest when three things coincide: the AI output informs a decision about real money or real people, the customer reasonably believes the output represents your professional opinion, and the output is materially wrong or discriminatory. When all three are present, you are in the territory of contract claims, potential ICO or FCA scrutiny, and reputational damage that can outlast the original error by some distance.

Lower-risk territory is when AI stays internal and low-stakes. Drafting meeting summaries, brainstorming marketing ideas, cleaning up internal documents: all of these carry limited client exposure, provided nothing reaches a customer without a substantive human review. A clear disclaimer on a customer-facing tool also reduces the reasonable-reliance argument, though it does not eliminate it.

One risk that is easy to overlook is internal drift. The NCSC has flagged that staff over-reliance on AI outputs can lead people to treat confident-sounding suggestions as verified fact. A UK consultancy study found that AI tools performing well 18 months ago may be delivering poorer results today without anyone noticing, if performance is not monitored. Wrong advice gets logged, sent, and relied on before anyone catches it.

What do you put in place to stay on the right side of this?

You do not need a legal team or an AI governance committee to manage this sensibly. Three things reduce your exposure quickly: a one-page internal policy that names what AI can and cannot draft for clients, a sign-off rule requiring a named person to review AI output before it touches a customer, and a conversation with your insurer about whether professional indemnity and cyber cover extends to AI-related errors.

Red-list the uses where AI should not generate client-facing content without expert review: legal advice, contract clauses, regulatory interpretations, financial or tax advice, and anything that could affect a hiring decision. These need a competent person to own the content, not just approve the layout.

Amber-list where AI can draft but a named team member must review before sending: client emails, proposals, reports, marketing copy. The review should check facts, tone, and whether the promises made are ones you can actually keep.

On data protection, keep personal data out of public AI tools where you can. If your workflow requires it, put a data-processing agreement in place with the provider and verify that data is not used for model training. The ICO is clear that where AI output has a significant effect on an individual, that person must be able to contest the decision and receive an explanation. Getting this wrong is not a minor procedural issue.

On insurance, the conversation with your broker matters. Hiscox UK’s guidance flags that professional indemnity and cyber policies may exclude losses caused by unvetted use of third-party AI tools. That gap is worth closing before something goes wrong, not after.

Firms get into trouble with AI advice across the complexity spectrum. The Air Canada chatbot was handling routine queries. The problem was a working assumption that it probably knew what it was talking about. Checking that assumption takes an afternoon. Replacing a damaged client relationship takes considerably longer.

Sources

- ICO (2023). Guidance on AI and data protection. Confirms organisations using AI are data controllers accountable for accuracy, fairness and lawfulness of automated outputs, including outputs from third-party AI providers. https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/artificial-intelligence - ICO (2023). Automated decision-making and profiling. Sets out individual rights to contest automated decisions and the requirement for businesses to provide meaningful explanations where AI has a significant effect on a person. https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/guide-to-data-protection/key-dp-themes/rights-related-to-automated-decision-making-including-profiling - FCA (2024). AI Update: Guiding Principles on Artificial Intelligence in Financial Services. States that FCA-authorised firms remain fully responsible for consumer outcomes, including Consumer Duty obligations, regardless of whether AI tools are used. https://www.fca.org.uk/news/speeches/ai-update-guiding-principles-artificial-intelligence - NCSC (2023). Guidance on secure use of AI and large language models. Warns that staff over-reliance on AI outputs can introduce incorrect information into live systems and recommends treating AI suggestions as untrusted until verified. https://www.ncsc.gov.uk/guidance/secure-use-of-ai-and-large-language-models - Civil Resolution Tribunal, British Columbia (2024). Moffatt v Air Canada. Ruled that Air Canada was responsible for misleading fare information provided by its chatbot; the airline's argument that the chatbot was a separate legal entity from the company was rejected. https://decisions.civilresolutionbc.ca/crt/crtd/en/item/525077/index.do - ICO and Equality and Human Rights Commission (2023). Explaining decisions made with AI: equality and data protection. Joint guidance warning that AI used in hiring, credit or insurance must avoid unjustified biases against protected characteristics under the Equality Act 2010. https://ico.org.uk/about-the-ico/who-we-work-with/external-organisations/equality-and-human-rights-commission - BBC News (2024). Air Canada must honour refund given by chatbot, tribunal rules. Reports the international implications of the Air Canada liability ruling for businesses using AI-powered customer service. https://www.bbc.co.uk/news/world-us-canada-68239471 - Vodafone Business (2024). Disadvantages of Artificial Intelligence for business. Cites AI hallucination rates between 3% and 27%, consumer frustration statistics showing 50% of consumers frustrated by chatbot interactions, and the security challenges created by AI adoption. https://www.vodafone.co.uk/business/sme-business/small-business-advice/ai-disadvantages-small-business - Hiscox UK (2024). AI and professional indemnity: what businesses need to know. Notes that professional indemnity and cyber policies may exclude losses caused by unvetted use of third-party AI tools or by breaches of confidentiality obligations. https://www.hiscox.co.uk/business-blog/ai-professional-indemnity - LegalVision UK (2024). Legal Implications of Using AI as a Small Business Owner. UK-focused legal commentary confirming business owners are responsible for chatbot statements and that AI-generated legal documents may not meet UK GDPR requirements. https://legalvision.co.uk/data-privacy-it/ai-small-business

Frequently asked questions

If my AI chatbot gives a customer wrong information, am I legally responsible?

Yes, under UK law. AI tools are not legal persons, so any claim from a customer lands on the business that deployed the chatbot. Many vendors disclaim accuracy in their terms of service, pushing responsibility back to you. The Air Canada chatbot case in 2023 confirmed this in published case law. If the wrong information led to a financial loss or a decision the customer would not otherwise have made, your business is in the frame.

What is the ICO's position on businesses using AI to make decisions about customers?

The ICO confirms that organisations deploying AI are data controllers responsible for the accuracy, fairness, and lawfulness of automated outputs. Where AI makes decisions that have a significant effect on an individual, such as credit, hiring, or affordability assessments, the individual has the right to contest the decision and receive an explanation. Getting this wrong can breach the accuracy and fairness principles in UK GDPR, exposing the business to enforcement action.

Do I need a formal AI policy before using AI tools in my business?

Not in the sense of a complex legal document. A one-page internal policy that names what AI can and cannot draft for clients, requires a named sign-off for anything customer-facing, and tells staff to treat AI output as a first draft rather than a finished answer is enough to start. Review your professional indemnity and cyber insurance policies to check whether AI-contributed errors are covered, and put a data-processing agreement in place with any AI tool that handles personal data.

This post is general information and education only, not legal, regulatory, financial, or other professional advice. Regulations evolve, fee benchmarks shift, and every situation is different, so please take qualified professional advice before acting on anything you read here. See the Terms of Use for the full position.

Ready to talk it through?

Book a free 30 minute conversation. No pitch, no pressure, just a useful chat about where AI fits in your business.

Book a conversation

Related reading

If any of this sounds familiar, let's talk.

The next step is a conversation. No pitch, no pressure. Just an honest discussion about where you are and whether I can help.

Book a conversation