The short answer
Picture a new AI assistant whose prompts, outputs and logs each end up on a different company’s servers. Your HIPAA risk analysis has to follow patient information to every one of those places. Do it per tool: map where the data goes, list what can go wrong, decide what to do, and review the logs.
HIPAA’s Security Rule requires covered entities and business associates to conduct “an accurate and thorough assessment of the potential risks and vulnerabilities to the confidentiality, integrity, and availability” of the ePHI they hold, then implement measures that reduce those risks to a reasonable and appropriate level.1 For an AI tool, do it per tool: map every place ePHI goes (including prompts, outputs, logs, caches and model providers), list threats including AI-specific ones such as prompt injection and wrong writes, score likelihood and impact, decide what to do, and review system activity regularly. NIST SP 800-66 Rev. 2 is a practical guide to the whole rule.3 This isn’t legal advice.
Deedglass
A sealed record you can check offline
Record 1043
- What ran
- Medication order, Bed 12
- Approved by
- Dr. N. Shah
- Check
- Matches
- Time
- 09:12
- The request
- Not keptA fingerprint only: 77d2…c41a
Each record is linked to the one before
- Record 1041
- Record 1042
- Record 1043
What the rule requires
Under 45 CFR 164.308(a)(1), the security management process includes four required specifications:1
| Specification | What it asks |
|---|---|
| Risk analysis | Accurate, thorough assessment of risks to ePHI confidentiality, integrity and availability |
| Risk management | Measures that reduce risks to a reasonable and appropriate level |
| Sanction policy | Sanctions for workforce who break security policies |
| Information system activity review | Regularly review records such as audit logs, access reports and incident reports2 |
The technical safeguards add audit controls: mechanisms that record and examine activity in systems holding ePHI.4 An AI tool is such a system the moment ePHI enters it.
Note “integrity”. Most AI risk discussions focus on confidentiality, data leaking. An AI tool that writes wrong data into a record is an integrity risk, and the rule covers it equally.
Step 1: scope every flow
Draw the data flow for the tool. For a typical AI tool:
Code What the screen or file shows, copied as is.
EHR ──▶ connector ──▶ app server ──▶ model provider API
▲ │ │ │
└── writes back ────────┘ ├─ logs └─ provider logs/retention
├─ cache
└─ analytics
For each box and arrow, ask: does ePHI pass or rest here, who operates it, and is it covered by an agreement? Vendors and their subcontractors that handle ePHI are generally business associates.5 Common misses: prompt logs, error traces, analytics events, vector stores, and the model provider’s own retention.
Step 2: list threats, including AI ones
Start from your usual threat list (lost devices, phishing, misconfiguration), then add:
| AI threat | Affects | Example |
|---|---|---|
| Prompt injection via documents or messages | Confidentiality, integrity | A referral letter tells the assistant to send the chart elsewhere |
| Over-broad tool permissions | All three | An agent with write access to every resource type |
| Wrong writes reported as success | Integrity | A field dropped on write, logged as done |
| Model output errors | Integrity | Fabricated or omitted details in drafts6 |
| Data retained or used by the model provider | Confidentiality | Prompts kept beyond contract terms |
| Model or version change | Integrity, availability | Behaviour shifts after an unannounced update |
| Vendor outage | Availability | Clinic workflow stops when the API is down |
The model output row has evidence behind it. A pilot published in 2026, reviewing ambient scribe notes, found omissions in 18% and hallucinations in 11.5%.6 That’s a single study, not a rate for every tool, but it shows integrity risk is real.
Step 3: score and decide
Use a simple scale and be consistent:
| Likelihood × Impact | Low impact | Medium | High |
|---|---|---|---|
| Likely | Medium | High | High |
| Possible | Low | Medium | High |
| Unlikely | Low | Low | Medium |
For each risk: accept, reduce, transfer (contract or insurance) or avoid (don’t do the thing). Record who decided, when and why. The rule asks for measures that bring risk to a reasonable and appropriate level1, so the reasoning matters as much as the score.
An example register for an AI scribe
This is an illustration for a hypothetical clinic, not a template to copy unchanged.
| # | Risk | L | I | Rating | Treatment | Owner |
|---|---|---|---|---|---|---|
| 1 | Audio and transcripts retained by vendor beyond need | Possible | High | High | Contract retention limit; verify deletion | Privacy officer |
| 2 | Draft notes contain errors that get signed | Likely | Medium | High | Review routine; monthly draft-vs-signed sampling | Clinical lead |
| 3 | Queued orders accepted without review | Possible | High | High | Orders as drafts; clinician signs each | CMIO |
| 4 | Model provider changes model without notice | Possible | Medium | Medium | Contract notice clause; regression test set | IT |
| 5 | Prompt logs hold ePHI in a third-party tool | Likely | Medium | High | Remove or redact; restrict access | IT security |
| 6 | Vendor API outage | Possible | Low | Low | Fallback to manual notes | Practice manager |
| 7 | No record separating AI draft from clinician edits | Likely | Medium | High | Require export of drafts and diffs | HIM |
Seven rows is a start, not an end. Each tool will have its own.
Step 4: review and repeat
Information system activity review is required: regular review of audit logs, access reports and incident tracking.2 For AI tools, review:
- What the tool accessed and wrote, by volume and by type.
- Mismatches between what it reported and what was stored.
- Approvals that took seconds, and writes that skipped approval.
- New subcontractors or model versions.
Redo the analysis when the tool changes materially: new write permissions, a new model provider, a new data type such as images. Keep each version; the history shows your reasoning over time.
Questions people ask
Does each AI tool need its own risk analysis?
The rule requires an accurate and thorough analysis of risks to the ePHI you hold.1 A tool with its own data flows and threats needs its own section at least.
Is there an official template?
NIST SP 800-66 Rev. 2 offers guidance and resources for implementing the Security Rule.3 Adapt it, adding AI threats.
Who should do it?
Your security official, with the tool’s owner, a clinical lead, the privacy officer and the vendor’s input on its data flows.
For how agent writes can be held, checked and recorded, see our healthcare page and the provenance spec; the FHIR write integrity study shows the integrity risk in numbers. To try approvals and read-backs on a real database in your browser, open the playground; to grade what an agent can reach on your own machine, run npx deedglass scan (the grade, explained).
Written with AI assistance and checked against our claims register.