What Good AI Incident Management Looks Like

An AI system begins producing unusual decisions. The technical team fixes the issue within hours and service returns to normal. Yet no one can say exactly when the failure started, which model version was running, how many people were affected, why the issue was not escalated sooner, whether a notification obligation was triggered, or whether the same failure mode exists elsewhere.

The technical problem is over. The governance problem is not.

Good AI incident management is not a ticket queue for moments when a model “acts strangely.” It is an end-to-end process connecting detection, classification, decision rights, containment, investigation, communication, recovery, and learning. For internal audit, the real question is not merely whether the organization can fix an AI failure. It is whether management can demonstrate that the response was timely, proportionate, traceable, and capable of reducing recurrence.

AI incidents are not just system outages

Traditional technology incidents are often associated with downtime, cyberattacks, or data loss. An AI system can fail while remaining technically available. It can operate at 99.9% uptime while producing discriminatory outcomes, degrading as production data changes, exposing sensitive information, generating materially incorrect content, or encouraging users to make poor decisions.

The NIST AI Risk Management Framework treats response and recovery as part of ongoing AI risk management. Its Manage function covers post-deployment monitoring, incident response, recovery, change management, and documentation. The NIST Generative AI Profile goes further by recommending that organizations record and track errors, near misses, and negative impacts, perform after-action reviews, and communicate incidents to relevant AI actors.

That changes the working definition of an incident. A mature organization does not wait for confirmed harm. It also establishes criteria for capturing near misses, threshold breaches, unexpected outputs, control failures, loss of effective human oversight, third-party failures, and repeated signals that may indicate a systemic problem.

The time to decide what counts as an AI incident is before the crisis, not during it.

Triggers and escalation should be designed in advance

The first weakness an auditor should look for is a vague trigger. Phrases such as “escalate when the impact is significant” sound reasonable until nobody can explain what significant means.

Triggers should be tied to the system's risk profile. They may include performance outside approved limits, data-quality failures, potentially harmful outputs, policy violations, loss of human-oversight controls, privacy or security events, material complaints, vendor failures, or unexpected behavior following a model or configuration change.

Classification should consider more than technical severity. Relevant factors include impact on people, scale, reversibility, duration, regulatory or contractual exposure, data sensitivity, potential propagation across other systems, and uncertainty. An incident with uncertain impact may require faster escalation, not slower escalation.

Escalation also needs named decision makers and expected response times. For each severity level, the organization should know who can suspend the system, who leads the response, when Legal, Risk, Compliance, Privacy, or Cybersecurity must join, who informs senior management, and which events reach the board or a designated committee.

The IIA's AI Framework Support Tools provide a useful audit-oriented reference point. Its Management AI Risk and Control Questionnaire presents, explicitly as example practices rather than mandatory requirements, an AI incident process covering detection, logging, classification, investigation, root-cause analysis, remediation, lessons learned, and escalation of material incidents.

Strong response separates containment, investigation, and remediation

One of the most common mistakes is treating “fixing the model” as synonymous with “managing the incident.” The activities are related, but their purposes are different.

Stage	What good looks like	Evidence an auditor can expect
Containment	Limit harm by suspending a feature, rolling back a version, invoking a manual fallback, restricting users, or isolating an integration.	Decision time, owner, approval, affected version, temporary controls.
Investigation	Reconstruct what happened, who was affected, and why; preserve records before changing the environment.	Timeline, logs, model and version, relevant data, prompts or settings, causal analysis, failed controls.
Remediation and recovery	Correct root and contributing causes, validate the fix, and formally authorize return to operation.	Action plan, test results, restart approval, heightened monitoring, closure evidence.

The distinction matters because a fast fix can destroy evidence. If the team overwrites the model version, changes reference data, or removes logs before preserving the state of the system, it may later be impossible to determine the true scope or cause of the incident.

ISO/IEC 42001:2023 contributes a management-system logic rather than a universal incident playbook. The standard requires organizations to determine what must be monitored and measured and, when a nonconformity occurs, to react, control and correct it, evaluate causes, implement corrective action, and review effectiveness. Its reference controls and implementation guidance also address external reporting and communication of incidents. Those controls sit within a risk-treatment approach; they do not impose one identical incident workflow on every organization or AI system.

Reporting is a governance decision, not an improvised email

An incident may require communication to different audiences: the process owner, AI system owner, second-line functions, senior management, the board, a vendor, customers or affected individuals, and potentially a regulator.

The incident playbook therefore needs a notification matrix that answers four questions: what event triggers communication, who decides whether it is reportable, who must be informed, and by when. Legal and Compliance should validate that decision logic for the organization's jurisdictions and AI use cases.

Regulation (EU) 2024/1689 illustrates why a universal deadline is a poor design choice. Article 73 establishes serious-incident reporting rules for providers of high-risk AI systems, while the current application timetable phases the relevant Chapter III provisions in on different dates depending on how the high-risk system is classified. The broader control lesson is more important than the European rule itself: regulatory notification should be an explicit decision point in the incident workflow, with evidence of who assessed the obligation, on what facts, and with what conclusion.

Internal reporting should also move beyond incident counts. Boards and executive committees need trends: incidents by type and severity, time to detect and contain, recurrence, common root causes, concentration by system or vendor, overdue corrective actions, and exceptions to approved risk appetite.

What internal audit should be able to reconstruct

The most revealing test is not asking whether a policy exists. It is selecting actual incidents and reconstructing them from first signal to closure.

An auditor should be able to follow the full chain: when the alert appeared, who received it, how it was classified, what criterion triggered escalation, who authorized containment, what evidence was preserved, how scope was determined, which communication obligations were assessed, how the fix was validated, and what changed afterward.

Useful evidence includes the incident and near-miss register, severity criteria, monitoring alerts and thresholds, responsibility matrices, system logs and versions, root-cause reviews, notification decisions, corrective-action plans, recovery testing, third-party incident clauses, and periodic reporting to governance bodies.

The process should also be tested before the next real failure. A tabletop exercise can introduce an unexpected scenario — for example, a vendor changes an underlying model and materially different outputs emerge — and reveal whether teams know who decides, what gets stopped, what evidence must be preserved, and who needs to be told. The NIST Generative AI Profile specifically recommends rehearsing response plans for third-party GAI technologies and improving them through retrospective learning.

Technical closure is not the end of the incident

Mature incident management ends when the organization has learned, not when the system is available again. A post-incident review should ask what conditions allowed the event, why preventive or detective controls did not work, whether similar systems are exposed, and what must change in the AI inventory, risk assessment, monitoring thresholds, control design, contracts, training, or human oversight.

NIST's 2026 report on monitoring deployed AI systems reinforces an important point: real-world AI behavior can differ from pre-deployment evaluation, and post-deployment monitoring practices are still developing. Incident management therefore cannot be designed as a static procedure.

For internal audit, the conclusion is straightforward. An organization does not demonstrate resilience because it has closed a few AI incidents. It demonstrates resilience when it detects earlier, decides against defined criteria, preserves evidence, communicates on time, recovers under control, and turns each incident into stronger controls for the next cycle.