An employment tribunal claim is not the kind of thing many founders expect to encounter after subscribing to an HR software tool. But that is increasingly the shape of the risk. You use a platform to help screen CVs. The algorithm does the ranking. A candidate does not make the shortlist. They ask why. You cannot tell them, because you do not know exactly how the model works. They go to a tribunal.
That scenario is not theoretical. In 2018, Amazon abandoned its own internal AI recruiting tool after discovering the system had been downgrading applications from women, trained as it was on a decade of hiring data that skewed heavily male. Amazon scrapped the project when the bias was found. Many businesses will not catch it at all.
What does AI discrimination risk mean in an employment context?
Under the Equality Act 2010, employers are liable for discriminatory outcomes in hiring, promotion and dismissal regardless of how they were produced. If an AI tool shortlists candidates in a way that disproportionately excludes people by age, sex, race or disability, the employer faces the same potential claim they would if a human manager had made the same call.
The law covers nine protected characteristics, including age, disability, race, sex, religion and sexual orientation. Discrimination does not have to be intentional. Indirect discrimination, where an apparently neutral process disproportionately disadvantages a protected group, is sufficient for a claim. A hiring algorithm trained on historical data from a business where senior hires have been predominantly from one demographic will tend to reproduce that pattern, and that tendency can constitute indirect discrimination even when no one designed it that way.
The ICO has identified several causes of discriminatory AI outcomes: training data that under-represents certain groups, historical data reflecting past discriminatory practices, and model design that optimises for overall accuracy at the expense of fairness for minority groups. Bias at any one of these stages can produce discriminatory outcomes in your hiring process.
Why does liability sit with you, not the software vendor?
Buying a recruitment platform that uses AI does not transfer your legal exposure to the vendor. The law locates liability with the employer, who remains responsible for outcomes produced on their behalf. If the ICO or an employment tribunal wants to know why a protected group was systematically excluded, they will ask you, not the software company that built the scoring model.
There is a specific data protection dimension here. Under UK GDPR, as updated by the Data Protection and Digital Information Act 2024, decisions with significant effects on individuals, including job rejection, must not be taken solely by automated means without meaningful human review. The employer must be able to explain what happened and show that a human was genuinely involved in reaching the outcome, not just confirming what the algorithm already decided.
Courts have added nuance to what counts as an automated decision. A 2023 ruling by the Court of Justice of the EU indicated that an AI-generated ranking or score that effectively determines who progresses may itself count as an automated decision, even when a human nominally reviews the list before issuing invitations. Employment lawyers in the UK are already applying that interpretation when auditing automated hiring workflows. If your process is “AI filters, then manager glances at the shortlist”, you may not have the human oversight the law expects.
Where in an owner-managed business does this exposure actually appear?
The risk concentrates wherever AI influences a decision with a legal or significant effect on an individual. For an owner-managed business in practice, that means CV screening and candidate ranking, performance scoring systems that influence pay reviews or disciplinary decisions, and automated monitoring of remote workers. The further the AI output is from a documented human decision, the higher the exposure.
The ICO’s guidance singles out AI recruitment tools as a high-risk area, and expects organisations using them to test for bias before deployment, maintain meaningful human involvement in significant decisions, and be ready to explain outcomes to candidates who ask.
The pattern of enforcement is clearest in the US so far. The Equal Employment Opportunity Commission secured a settlement with iTutorGroup in 2023 after its software allegedly auto-rejected female applicants aged 55 and over and male applicants aged 60 and over. The $365,000 settlement was the EEOC’s first AI discrimination enforcement action. UK and EU regulators are watching.
The question to ask about any platform is not whether it uses AI, but whether you understand what signals it acts on and whether those signals could correlate with protected characteristics. A tool that scores candidates on “cultural fit” based on historical hiring data is a good example of one where the risk warrants scrutiny.
When should you treat this risk as active versus theoretical?
The risk is active whenever your business uses AI to rank, score, screen or flag candidates or employees for significant decisions, and when you cannot trace how those outputs were reached. A firm that hires one or two people a year through manual shortlisting faces a different exposure than one running a hundred applications through an automated scoring tool.
Three conditions raise exposure significantly: the AI makes or strongly influences a decision without a clear human override; you cannot explain the outcome to a rejected candidate; or your hiring data shows a pattern of skewed outcomes between demographic groups without a documented and justifiable reason.
Where AI assists with lower-stakes tasks, drafting job adverts, summarising interview notes, scheduling, the risk is considerably lower. Those uses do not generate the “significant decision” footprint that triggers the tighter GDPR rules or the main equality exposure.
An annual outcome check, even an informal one, is becoming a baseline expectation. New York City requires independent bias audits annually for automated hiring tools under Local Law 144. The UK has no equivalent mandate yet, but the ICO and EHRC have both identified algorithmic bias in hiring as an active regulatory concern.
What legal frameworks are shaping this right now?
Three overlapping frameworks define the current compliance landscape for UK employers. The Equality Act 2010 is the primary route for discrimination claims. UK GDPR, as updated by the Data Protection and Digital Information Act 2024, governs automated decision-making rights. And the EU AI Act, applying from 2026 to 2027, classifies AI used in recruitment and worker management as high-risk.
The EU AI Act is worth noting even if you do not operate in the EU. Businesses that recruit for EU clients or use software built to EU specifications may find these obligations apply indirectly. Vendors are already marketing “AI Act ready” products, and that language should prompt you to ask for the underlying evidence rather than accepting the label.
A draft Artificial Intelligence (Regulation and Employment Rights) Bill, under discussion in Parliament, would go further, reversing the burden of proof so that employers must demonstrate their AI is not discriminatory, and requiring worker consultation before deploying high-risk AI in employment decisions. This is not law yet, but it reflects where UK policy is heading.
The practical point, drawn from the 2024 Industrial Law Journal analysis of UK equality law and AI, is that the current framework is largely reactive, relying on individuals to bring claims after harm occurs. Proactive auditing, documentation and human oversight are where businesses with genuine exposure should focus, rather than waiting for enforcement to arrive first.
