An incident involving AI is an incident like any other until the operator has to say what kind it is, who may stop it and which authority must be told by when. This chapter concerns O2 Diagnose and O4 Restore. It defines what counts as an AI incident for the operator, describes the signals that detect one, gives a triage tree that runs from symptom to class to owner to stop path, sets out containment by a stop path outside the system it stops, and puts the reporting clocks of the AI Act, NIS 2, DORA and the GDPR side by side so that one incident record can carry them all. It closes with the published incidents of 2025 and 2026 read by mechanism. The definitions of the five operations functions, of the autonomy budget with its action classes and of the observer principle are those of Chapter 4; this chapter applies them and does not restate them beyond what the argument needs.
The chapter builds on conclusions of the earlier volumes. AI Platform and Security, Chapters 17 and 21, added an AI classification step to incident management, an impact query against the event log and a path back to evaluation, and treated the reporting deadlines as design constraints on the logs (F46) (297, Nordqvist, 2026). AI Threat, Chapters 25 and 35, found that agents acting beyond their mandate are observable and that the incidents of 2025 and 2026 share a pattern of mechanism (F67) (299, Nordqvist, 2026). AI Blue, Chapter 19, showed that one incident opens several clocks and that the case record should carry one clock per regime, each with its own start, limit, owner and state (F106) (302, Nordqvist, 2026). The security view of the same incidents belongs to AI Blue; this chapter takes the operator's view, in which the first question is what has stopped working or stopped being trustworthy, and the reporting duties follow from the answer.
The OECD defines an AI incident as an event, circumstance or series of events in which the development, use or malfunction of one or more AI systems directly or indirectly leads to one of four kinds of harm: injury or harm to health, disruption of critical infrastructure, violations of human rights or breaches of law protecting fundamental, labour and intellectual property rights, and harm to property, communities or the environment. It defines an AI hazard with the same structure, where the system could plausibly lead to an incident (62, OECD, 2024). The AI Act uses a narrower and legally operative term. A serious incident is an incident or malfunctioning of an AI system that directly or indirectly leads to the death of a person or serious harm to health, a serious and irreversible disruption of the management or operation of critical infrastructure, the infringement of obligations under Union law intended to protect fundamental rights, or serious harm to property or the environment (Article 3, point 49) (1, European Union, 2024).
Primary topics explored
- What an AI incident is
- Detection
- Triage
- Containment
- Reporting clocks
- Published incidents
About AI Operations
AI Operations is Book 10 of The Transformed Architecture. It takes IT operations as its subject and asks what happens when AI takes part in it: the five operations functions (observe, diagnose, change, restore, sustain), the autonomy budget and the Level 4 prohibitions, the observer principle, resilience as a claim that needs evidence, operating AI systems, the operator's own AI as Tier 0 and how to defend it, operating models, decision rights, regulation (DORA, NIS 2, the AI Act), and the people involved, for European enterprises and public authorities.
It builds on the conclusions of Architecture as Code, AI Governance, AI Platform and Security, AI Red Team, AI Threat, The Transformed Security Architect, AI Boss, AI Blue and AI Purple.
CIOs and heads of IT operations, SRE and platform leads, incident commanders and on-call engineers, service-desk and service-management leads, FinOps and capacity managers, architects, CISOs and heads of resilience and third-party risk, boards and management bodies, procurement and vendor managers, supervisors and auditors, works councils and unions, data protection officers, and public-sector leaders.
Gunnar Nordqvist
Conceived, directed and published the books of The Transformed Architecture.
Continue with the full chapter
Chapter 14: Incidents Involving AI: Detection, Triage and Reporting in AI Operations gives the detailed explanations, practical examples and implementation patterns behind this summary.
This whitepaper belongs to “Operating AI Systems” and is one of 35 drawn from the manuscript.