The short answer
Picture discovering on a Monday that an AI tool has been writing wrong codes into charts since Friday. The first hour decides how much you can fix and how much you can show. Stop its writes, keep the evidence, find the damage from the records themselves, and check which reporting clocks have started.
When an AI tool writes wrong data into a hospital system, respond in this order: stop its writes, keep its reads only if you need them, preserve the evidence, scope the damage from the system of record rather than from the tool’s own log, correct the records in the source system with a person approving each fix, decide whether a reporting duty applies, and review what let it happen. NIST’s incident response guidance, SP 800-61 Revision 3 (April 2025), frames incident response as part of overall cybersecurity risk management under CSF 2.0.1 The specific clocks depend on what happened: CERT-In wants listed incidents reported within 6 hours2, and personal data breaches have their own timelines under India’s DPDP Rules and HIPAA.
This post is about systems and records. Patient safety decisions belong to clinicians and your clinical governance process.
Deedglass
Saved is not the same as saved correctly
Deedglass reads the order back
How an AI error differs from a breach
Most incident plans are written for attackers. An AI error often has no attacker: a tool that wrote to the wrong patient, dropped a field, duplicated records or marked something done that wasn’t. Three things make these different:
- The tool reports success. Errors don’t trip alarms; someone notices downstream.
- The blast radius is in the data, not the network. You’re looking for wrong records, not compromised hosts.
- The tool’s own log may be wrong. If the agent misreported what it did once, its log can’t be trusted to scope the damage.
Some AI errors are breaches, such as an agent sending records to the wrong recipient. Treat each one as possibly reportable until someone qualified decides otherwise.
The runbook
| Step | Goal | First actions | Owner |
|---|---|---|---|
| 1. Stop | No more wrong writes | Revoke or pause the tool’s write access; leave reads only if needed | IT on call |
| 2. Preserve | Keep evidence intact | Snapshot logs, tool config, prompts, versions; don’t “clean up” yet | IT security |
| 3. Scope | Know every affected record | Query the system of record for the tool’s writes in the window | HIM and IT |
| 4. Assess | Clinical and privacy impact | Clinicians review affected charts; privacy officer reviews disclosure | Clinical lead, DPO |
| 5. Correct | Fix the record | Amend in the source system; a person approves each fix; keep originals | HIM |
| 6. Notify | Meet any duty that applies | Check CERT-In, DPDP, HIPAA, contracts; record the decision | DPO, legal |
| 7. Learn | Stop a repeat | Root cause; change rules, scopes or review steps | Governance |
Stop means writes, not the whole tool, unless you can’t separate them. A hospital that can’t revoke an AI tool’s write access in minutes doesn’t have a stop button. Test it before you need it; our kill switch post covers how.
Scope from the system of record. Pull every resource the tool’s credentials created or changed in the window, from the EHR or FHIR server’s own history and audit trail, and compare with what the tool says it did. The difference is part of your finding.
Correct without deleting. Amend records so the wrong version stays in history with a reason. An AI-assisted bulk fix is a second incident waiting to happen; if you use one, put every change through approval and read-back.
The reporting clocks
| Regime | Who | Clock | Notes |
|---|---|---|---|
| CERT-In Directions (India) | Service providers, body corporates and others listed | Listed incidents within 6 hours | Also keep ICT logs for a rolling 180 days in India2 |
| DPDP Rules 2025 (India) | Data fiduciary | Affected people without delay; the Board without delay, details within 72 hours | Applies to personal data breaches3 |
| HIPAA (US), 164.410 | Business associate to covered entity | Without unreasonable delay, no later than 60 days after discovery | Breach of unsecured PHI4 |
Not every AI error triggers any of these. A wrong value written to the right patient’s chart, inside the hospital, may be a data-quality and safety event rather than a breach. Have your privacy officer and legal team make the call, and write down why.
India’s AI Governance Guidelines also suggest a national, federated database of AI incidents that organisations would be encouraged to report to without threat of penalties, using existing mechanisms such as CERT-In’s.5 That’s a suggestion, not a duty today, but keeping your AI incident records in a shareable shape costs little.
A worked example
This scenario is made up, to show the steps in order. At 09:10 a nurse notices a lab value in a chart that doesn’t match the lab system. The interface’s AI-assisted correction tool had “fixed” units on overnight results.
- 09:20 Stop. IT revokes the tool’s write scope. Reads stay on for the investigation.
- 09:30 Preserve. Logs, the tool’s config and model version, and its prompt templates are copied to a case folder.
- 10:15 Scope. The FHIR server’s history shows 214 Observations changed by the tool’s client id since 02:00. The tool’s own log lists 198. The 16-record gap goes into the finding.
- 11:00 Assess. Clinicians review the affected charts; the privacy officer confirms no data left the hospital.
- Afternoon, Correct. HIM restores the original values as amendments, approved one batch at a time and read back.
- Same day, Notify. Legal records that no breach notice applies, with the reasoning.
- Next week, Learn. Unit changes move to the approval tier; the tool gets a read-back step.
What to set up before the first incident
- A tested way to revoke each AI tool’s write access within minutes.
- Separate credentials per tool, so its writes can be found by client id.
- Records of each AI action that don’t depend on the tool’s own honesty: what it claimed and what the system shows.
- A named owner for each runbook step, with a phone number.
- A decision template for the reporting clocks, filled in for every incident.
Records signed and chained by a separate component help with scoping, though they show only what went through that component; a valid record means the issuer wrote those values and nothing changed since, not that every action was seen.7 The record spec describes the format.
Questions people ask
Is an AI writing wrong data a reportable breach?
Sometimes. If data reached someone who shouldn’t have it, breach rules may apply; if a wrong value stayed inside the right record, it’s usually a safety and quality event. Legal and privacy decide.
Should we turn the AI tool off entirely?
Stop its writes first. Keeping reads can help the investigation. Turn it off fully if you can’t separate the two.
Can we trust the tool’s own log to find affected records?
No. Use the system of record’s history and compare. In the example above, the gap between the two was itself a finding.
Try it. In Deedglass, lockdown refuses an agent’s writes in every session after an action fails its check, until a person acknowledges it6, and turning lockdown off needs two different admins.6 More on the hospital page. To try approvals and read-backs on a real database in your browser, open the playground; to grade what an agent can reach on your own machine, run npx deedglass scan (the grade, explained).
Written with AI assistance and checked against our claims register.