The clinical AI governance a delegate must get right before go-live

A professional reviewing documents at a desk in a clinical office with natural light
TL;DR

In a clinical business, AI governance is the licence to deploy. Four gates must be cleared before any AI tool touches patient data or informs a clinical decision. A signed Business Associate Agreement with every PHI-handling vendor, encryption at every stage, a documented clinician-review protocol, and recorded patient consent are the minimum, and all four come before go-live.

Key takeaways

- Any AI system that processes Protected Health Information requires a signed Business Associate Agreement with the vendor before deployment, regardless of how well the tool performs in a demo. - Routing patient data to a general-purpose AI model with no vendor agreement is a frequent source of HIPAA violations in small practices. A single misconfiguration of this kind can trigger regulatory action and void insurance cover. - AI output near a clinical decision counts as decision support. A clinician must review and sign off before that output influences care, under NICE guidance and equivalent professional standards. - Over 40 bills across 25 US states in 2026 have introduced requirements that patients know when AI is involved in their care. Patient consent disclosure is a live legal obligation in many jurisdictions, not a courtesy. - A right-sized governance record for a small practice covers vendor BAAs, a written clinician-review protocol, patient consent evidence, and a data flow map. It does not need to be complex to be effective.

The delegate handed an AI documentation project and told to get it live by quarter-end usually spends the first week looking at the technology. The BAA, the consent process, and the clinician-review protocol are the actual critical path, not the product demonstration. A clinical practice where someone routes patient data to an unvetted large language model sits one misconfiguration away from a notifiable breach, a voided insurance policy, and a regulatory investigation. Clinical AI governance is the licence to deploy, built before deployment begins, not a sign-off collected afterwards.

What is clinical AI governance?

Clinical AI governance covers the controls a clinical business needs before any AI system touches patient data or informs a clinical decision. The four minimum gates are a signed Business Associate Agreement with every vendor handling Protected Health Information, encryption at every stage of data transit and storage, a clinician-review protocol for AI output that could influence a clinical decision, and documented patient consent.

The distinction from generic AI governance matters. A payroll tool with a data-processing agreement and an audit log is, broadly, covered. A documentation tool transcribing patient consultations and feeding summaries into a clinical workflow sits in an entirely different risk class. HIPAA in the United States sets strict standards for the handling of Protected Health Information, and the US Office for Civil Rights enforces them. In the UK, the equivalent weight comes from the Data Protection Act 2018, NHS information governance requirements, and Care Quality Commission expectations.

The category of data drives the category of control. A business can deploy a general-purpose AI tool for admin with a light-touch agreement. When patient data enters that system, the light-touch agreement is insufficient regardless of how the vendor describes their platform.

Why does it matter more in a clinical business than anywhere else?

A scheduling error in a retail business is an inconvenience. A documentation error in a clinical practice can affect patient care. That asymmetry runs through every governance decision in a clinical setting. HIPAA carries financial penalties and potential clinical liability for failures that a signed vendor agreement would have caught. The same weight applies under NHS information governance frameworks. The risk class is genuinely different from anything else in an owner-managed business.

McKinsey’s Q4 2025 survey found that 50 per cent of healthcare leaders have implemented generative AI in their organisations, up from 25 per cent in Q4 2023. The adoption rate has outrun governance maturity in many practices. Healthcare leaders named inaccuracies in AI output, security vulnerabilities, and regulatory compliance failures as their primary concerns. The delegate asked to accelerate deployment is walking into exactly that gap.

The stakes are compounded because insurance cover depends on it. A practice that routes patient data to an unvetted AI model without a BAA has, in the eyes of many insurers, taken a deliberate action rather than made a negligent error. The implications for professional indemnity cover are significant.

Where will you actually meet the governance gates?

The BAA gate appears the moment you sign up to any tool that receives audio or text from a clinical consultation. The encryption gate applies as soon as patient data leaves your system. The clinician-review gate is live on the first patient note the AI generates. The consent gate should be in place before the others. Patients have the right to know when AI is involved in their care.

A frequently cited breach vector in small clinics is someone routing patient data to a general-purpose model with no vendor agreement in place. DoctorConnect’s guidance on healthcare AI integration notes that a single misconfiguration of this kind can trigger a HIPAA violation with severe financial and reputational consequences. The tool may work well, the output may be accurate, and nobody intends any harm. The absence of a BAA is the problem, regardless of intent.

The clinician-review requirement deserves specific attention. NICE’s guidance on AI-derived software in clinical settings is explicit that healthcare professionals must review AI outputs and that centres should maintain existing protocols alongside any AI tool. Healthcare professionals should be cautious when acting on software outputs without independent clinical review. The principle extends to a small practice deploying a documentation system. AI generates a draft note; the clinician reviews and signs it off under their professional responsibility.

Patient consent is now a live legislative front. Over 40 bills have been introduced across 25 states in 2026, according to Manatt Health’s AI Policy Tracker, with a consistent requirement that patients are aware when AI tools are involved in their care. A practice that does not disclose AI involvement cannot defend compliance by pointing to intent.

When do these requirements apply and when can you move faster?

The governance gates apply whenever AI touches Protected Health Information or could influence a clinical decision. A scheduling tool that never sees patient records sits outside the gate. An AI drafting appointment reminders with patient names and conditions does not. The line is the data, not the task. If the system processes PHI or could infer it from inputs, the full gate set applies.

The practical implication for a delegate managing several AI tools is that they fall into one of two categories. Admin-only tools with no access to patient records can be procured under a standard data-processing agreement, with appropriate staff training and access controls. Clinical-adjacent tools, anything that handles PHI or feeds into a clinical workflow, need the full gate set before the first patient interaction.

The categorisation matters because it protects the tools you can move quickly on. If everything is treated as clinical-adjacent, the governance overhead becomes unmanageable for a 90-person practice. If the distinction is drawn clearly and documented, admin tools can proceed while the clinical governance gates are being arranged for the higher-risk deployments.

What else belongs in a right-sized governance record?

A governance record for a 90-person practice can be brief, provided it covers what matters in a regulatory investigation. A log of vendor BAAs and their review dates, a written clinician-review protocol with named approvers, an evidence trail of patient consent disclosure, and a data flow map showing where PHI enters and exits each AI system. A single spreadsheet and two short documents will handle all four items.

What kills governance records in small practices is the enterprise template that arrives with 40 fields nobody fills in. The record needs to be light enough to maintain on a quarterly cycle by whoever holds clinical compliance responsibility. ASHA’s guidance for clinicians is clear that professionals retain responsibility for AI outputs and must be able to demonstrate that review happened. A governance record is how a practice proves it.

A quarterly review covers three things. First, confirm that all active vendors have current BAAs and check the review dates. Second, verify that the clinician-review protocol reflects how the tools are actually being used, not how they were expected to be used at go-live. Third, update the consent disclosure if any new AI system has been added to the clinical workflow. The practice that does this consistently holds the evidence if it is ever needed.

Getting the clinical boundaries in place before a tool goes live protects the AI programme. The BAA, the clinician-review protocol, and the patient consent disclosure take time to arrange correctly, and none can be retrofitted cleanly once a tool is handling patient data. A quarter-end deadline does not override a notifiable breach threshold. The delegate who treats governance as the pre-deployment gate, rather than the post-deployment tidy-up, is the one who keeps the programme in play.

Sources

- US Department of Health and Human Services, Office for Civil Rights (2024). HIPAA Security Rule. Federal standards for protecting electronic Protected Health Information including encryption in transit and at rest, role-based access controls, and full audit trails. https://www.hhs.gov/hipaa/for-professionals/security/index.html - US Department of Health and Human Services, Office for Civil Rights (2024). Sample Business Associate Agreement Provisions. Guidance on BAA requirements under HIPAA for vendors and AI systems that handle Protected Health Information. https://www.hhs.gov/hipaa/for-professionals/covered-entities/sample-business-associate-agreement-provisions/index.html - National Institute for Health and Care Excellence (2025). Recommendations on AI-derived software for CT brain scan review in stroke care. Mandates that clinical professionals review AI outputs and that centres maintain existing scan reporting protocols alongside AI tools. https://www.nice.org.uk/consultations/2357/1/recommendations - Manatt Health (2026). Health AI Policy Tracker. Tracks state legislation on clinical AI oversight and patient consent. Over 40 bills introduced across 25 US states in 2026 requiring clinical oversight and patient awareness of AI use. https://www.manatt.com/insights/newsletters/health-highlights/manatt-health-health-ai-policy-tracker - McKinsey (2025). Generative AI in healthcare: current trends and future outlook. Q4 2025 survey finding 50 per cent of healthcare leaders have implemented generative AI, up from 25 per cent in Q4 2023. Risk and safety cited as primary roadblock by 43 per cent of respondents. https://www.mckinsey.com/industries/healthcare/our-insights/generative-ai-in-healthcare-current-trends-and-future-outlook - American Speech-Language-Hearing Association (2024). Generative AI for clinicians. Professional guidance confirming clinicians retain responsibility for AI-generated content and must exercise professional judgement on when and whether to use AI tools in clinical workflows. https://www.asha.org/practice/generative-artificial-intelligence-for-clinicians/additional-ai-resources-and-guidance/ - CrossML (2024). AI compliance with HIPAA, GDPR and SOC 2. Covers BAA requirements, encryption standards, role-based access controls, and audit trail obligations for healthcare AI deployments. https://www.crossml.com/ai-compliance-with-hipaa-gdpr-and-soc2/ - DoctorConnect (2024). Healthcare AI APIs: integration, compliance and practical guidance. Flags the single-misconfiguration breach risk when PHI is routed to AI systems without a signed BAA in place. https://doctorconnect.net/healthcare-ai-apis-integration-compliance-and-practical/ - US Food and Drug Administration (2024). Artificial intelligence and machine learning in medical devices. Regulatory framework for AI and ML-enabled clinical decision support software, including oversight requirements and safety standards. https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-and-machine-learning-aiml-enabled-medical-devices

Frequently asked questions

Do I need a Business Associate Agreement for every AI tool my practice uses?

A BAA is required for any vendor whose system processes, stores, or transmits Protected Health Information. If an AI tool only handles scheduling data with no patient identifiers, a BAA may not be required. If the tool receives consultation audio, dictation, or any record linked to a patient, a BAA is mandatory before use. Check the vendor's HIPAA compliance documentation before sign-up.

What does clinician sign-off on AI output actually mean in a small practice?

It means a qualified clinician reviews any AI-generated clinical note, recommendation, or output before it enters the patient record or influences care. The clinician confirms accuracy, corrects errors, and approves the content under their professional responsibility. NICE guidance on AI-derived software makes clear that healthcare professionals must retain oversight and that practices should maintain existing reporting protocols. The AI generates a draft; the clinician owns the record.

Are the patient consent requirements the same in the UK as in the US?

The frameworks differ but the principle is consistent. In the US, over 40 bills across 25 states in 2026 have introduced requirements that patients know when AI is involved in their care, according to Manatt Health's AI Policy Tracker. In the UK, NHS information governance and data protection obligations require transparent disclosure of how patient data is used. A clinical practice in either jurisdiction needs a documented consent disclosure process before using AI in clinical workflows.

This post is general information and education only, not legal, regulatory, financial, or other professional advice. Regulations evolve, fee benchmarks shift, and every situation is different, so please take qualified professional advice before acting on anything you read here. See the Terms of Use for the full position.

Ready to talk it through?

Book a free 30 minute conversation. No pitch, no pressure, just a useful chat about where AI fits in your business.

Book a conversation

Related reading

If any of this sounds familiar, let's talk.

The next step is a conversation. No pitch, no pressure. Just an honest discussion about where you are and whether I can help.

Book a conversation