The First 48 Hours of an AI Incident
The short answer
The first hour and the next forty-seven are different jobs
Topic 3.5 wins the first hour with instinct trained into procedure: detect, declare, contain, communicate, log. This topic wins the next two days with discipline: classify against a legal standard, choose deliberately among real containment options, preserve evidence before your own response destroys it, and account to every party actually owed an account.
What you will be able to do
- Classify the severity of a completed or ongoing AI incident against a defined harm standard, not a gut feeling, distinguishing an operational bad night from a "serious incident" in the legal sense that a notification duty attaches to.
- Choose among the real containment options for an AI system, rollback, throttle, and disable, and justify the choice against the severity, the reach, and the cost of each option, rather than defaulting to the most dramatic or the most convenient one.
- Build an evidence-preservation checklist for the first 48 hours that names exactly what must survive a containment action (the model or configuration version, the affected inputs and outputs, the logs, the change history) and why deleting or "cleaning up" any of it during the response is itself a governance failure.
- Apply the EU AI Act's Article 73 serious-incident reporting framework to a real or plausible AI system: the definition of a serious incident, the tiered clocks, and who reports to whom, distinguishing the legal reporting duty from the customer-facing disclosure decision covered elsewhere (see Topic 3.7) (see Topic 12.3).
- Recognize that reporting obligations stack rather than replace one another: a single incident can trigger an AI-specific notification, a personal-data-breach notification, and a contractual notice to an insurer or a business partner, on different clocks, to different recipients, at once.
- Write a one-page leadership briefing for three checkpoints of the first 48 hours (hour one, hour eight, hour forty-eight) that gives a board or executive what they actually need to decide, not a narrative of the night.
- Distinguish the severity classification in this topic from the operational severity ladder built in Topic 3.5, and explain why an incident can be operationally minor (a SEV-3 on your own scale) and still meet a legal definition of "serious," or the reverse.
- Defend a containment and notification decision under challenge, in the language a regulator, an auditor, or a board member would use, by pointing to the classification, the evidence, and the clock rather than to intuition.
- Scope a notification matrix across jurisdictions for an AI system that serves users in more than one country, recognizing that a single incident can implicate more than one authority's law at once, and separating the legal determinations that need a licensed professional's confirmation from the classification work a governance professional can and must do first.
The ideas, one by one
Legal severity is not operational severity
Your own severity scale answers how urgently to move; the legal classification asks whether what happened meets a defined standard, death or serious harm to health, serious critical-infrastructure disruption, a fundamental-rights infringement, or serious harm to property or the environment. The two can diverge in either direction, and a mature response checks both, every time.
The containment menu has three real options, not one
Rollback undoes a specific, identifiable cause. Throttle narrows what a system is allowed to do without shutting it down, and is under-used relative to how often it is the proportionate choice. Disable is the heaviest and correct option when the cause is not yet known or the harm is severe; the question it raises is how long to hold, not whether to use it.
Containment can destroy the evidence a later review needs, unless you preserve first
The model version, the actual inputs and outputs from the incident window, the surrounding logs, the change history, and the decision log itself must survive your own rollback, restart, or cleanup, guarded by a named evidence owner and a paused deletion schedule, or they will not survive by default.
Notification obligations stack; they do not substitute for one another
A single incident can trigger an AI-specific report, a personal-data-breach report, a contractual notice, an internal leadership briefing, and a customer disclosure decision, on different clocks, to different recipients, all at once. Checking one and stopping is an incomplete answer, not a complete one.
The Article 73 clock starts on awareness, not on certainty
Two days for the gravest cases, ten days when the incident involves a death, no later than fifteen days after becoming aware for the general case, all three tiers codified directly in the statute. "Becomes aware" means recognizing a plausible serious incident, not finishing an investigation of one; waiting for certainty is how the deadline gets missed while the investigation is still, technically, in progress. A preliminary, honestly incomplete report is the normal and correct pattern.
Deployers have their own, faster duty
A deployer of a high-risk system who becomes aware of a serious incident must immediately inform the provider and the relevant authority, and cannot simply wait for the vendor to report. Knowing where your organization sits, provider or deployer, decides whose clock you are actually on.
Leadership needs decisions, not narrative, at three checkpoints
Hour one (what we know, what we did, what we do not yet know), hour eight (classification, containment logic, notification posture), and hour forty-eight (the defensible, complete record). Each is a fill-in-the-blank template written in advance, not composed under pressure.
The honest exception is to report provisionally, not to wait for certainty
When classification genuinely cannot be completed within the window, state the uncertainty in every briefing, report where a duty's trigger is plausibly met, and keep the evidence-preservation discipline running for as long as the uncertainty lasts.
A deferred legal deadline is not a deferred safety habit
The Digital Omnibus package moved the EU AI Act's high-risk application dates to 2 December 2027 (Annex III) and 2 August 2028 (Annex I), but the classify-choose-preserve-account discipline is worth building now, for every AI system that can hurt someone, regardless of which regulation attaches when.
Pak'nSave is the case for what happens without this discipline, not because it broke a law
New Zealand had no AI-incident-reporting statute for Foodstuffs North Island to violate, and the company still ran a response with no visible classification, no stated containment logic beyond an eventual quiet throttle, and no account of how many of its 35,000-plus sessions produced something dangerous. The four verbs, classify, choose, preserve, account, are worth running whether or not a specific law compels the last one.
This playbook is what the rest of your incident dossier depends on being true
The evidence this topic protects is what Topic 3.6's review reconstructs from, what Topic 3.7's customer letter is checked against, and what a Module 5 conformity file assembles into a record that survives a regulator's challenge. Run the first 48 hours with discipline, and every later step inherits an honest foundation.
Read the deadline backward from the earliest plausible awareness, not forward from a completed investigation
The tightest applicable clock should be checked against the moment your team first plausibly suspected a serious incident, not the moment a full investigation concluded; a playbook that assumes it can finish its own analysis before the clock starts has misread how the clock works.
A single system can cross borders, and the notification matrix has to follow
A system serving users in more than one country can implicate more than one authority's AI-specific, data-protection, or sector-specific notification law from the same incident. Build the matrix jurisdiction-aware in advance, and preserve evidence to the highest bar any applicable jurisdiction could require, not the lowest.
Listen to it
The same lesson, talked through by two hosts as episode 25 of the podcast.
Real cases
These examples sharpen one facet of the 48-hour discipline each. The deep anchor is Pak'nSave's Savey Meal-bot; several others are treated in depth by their owner topics and are referenced here only for the angle this topic needs.
Example 1 (the anchor): Pak'nSave's Savey Meal-bot and the response that never classified, chose, or accounted. In June 2023, Foodstuffs North Island launched Savey Meal-bot, an AI recipe generator built on a general-purpose language model with unrestricted free-text ingredient input. In early August, users discovered it would generate a "recipe" for chlorine gas from a bleach, ammonia, and water prompt, calling it an "aromatic water mix," alongside other dangerous suggestions including a "poison bread sandwich" (Gizmodo, 2023). Read against this topic's four verbs, the company's public response shows every gap this topic is built to close. Classify: the company's first public statement addressed how the tool had been misused, not what harm categories the output could plausibly have caused or to how many of the more than 35,000 recorded sessions (Newshub/Stuff, 2023). Choose: the eventual fix, restricting free-text input toward a curated ingredient list, is a genuine and correct throttle from this topic's menu (3C), but it arrived only after sustained public pressure, not as a considered decision made in the first 48 hours. Preserve: no public account exists of what evidence the company gathered about which users received dangerous outputs before the story broke, a gap that would matter enormously if any user had, in fact, attempted to follow one of the recipes. Account: with no AI-specific incident-reporting law in force in New Zealand at the time, there was no external notification clock to meet, but that also meant the only account offered was a reactive press statement, not a considered leadership briefing built from a completed classification. None of this makes Foodstuffs North Island unusually careless; it makes the company an ordinary one operating without the habit this topic teaches, exactly the gap a competency map identifies when an incident-response skill is thin across an entire market.
Example 2 (rollback done right, from this program's own record): Apple's Intelligence news-summary rollback. When Apple's AI-generated notification summaries were found to be inventing false headlines attributed to real news organizations, Apple monitored the feature, then disabled it, a containment choice this program has already treated in depth (see Topic 3.4). Read against 3C's menu, this is a defensible full-disable rather than a throttle, because the harm (fabricated content attributed to real publishers) was not cleanly traceable to a narrow input pattern the way Pak'nSave's chlorine-gas prompt was; the fabrication risk ran across the whole feature, which is exactly the condition 3C names for choosing disable over throttle.
Example 3 (a fast, mechanical clock that outran a slow human response): Knight Capital. This program's deep treatment of Knight Capital covers the first minutes of a machine-speed incident (see Topic 3.5). Read forward into this topic's window, Knight's forty-five-minute loss and its aftermath illustrate 3D's evidence-preservation point from a different angle: the SEC's findings relied heavily on records that did happen to survive, the automated alert emails, deployment logs, and order records, not because Knight had built a preservation discipline but because those particular records were not the ones anyone thought to delete. A firm that had also lost its logs in the scramble would have faced the same forty-five-minute catastrophe with no way to reconstruct what happened afterward, a considerably worse position than the one Knight was actually in.
Example 4 (an incident that met a legal notification bar, non-AI, cited for the clock mechanics): why "becomes aware" is the trigger, not "confirmed." Data-protection regulators enforcing GDPR's 72-hour breach notification clock have repeatedly rejected the argument that a company's internal investigation period excuses a late notification, holding instead that the clock starts when the organization has a reasonable degree of certainty a breach affecting personal data has occurred, not when the investigation concludes. The same "becomes aware, not becomes certain" logic is what 3F applies to Article 73, and it is worth naming as a general regulatory pattern rather than an AI-specific quirk: regulators across data protection and AI law alike have converged on the same answer to "when does the clock start," because the alternative, letting companies set their own start time by choosing how long to investigate before deciding, would make any fixed deadline meaningless.
Example 5 (a severity miss in the other direction): the Chicago Sun-Times AI-generated summer reading list. In 2025, a syndicated feature section published a summer reading list containing AI-generated, non-existent book titles credited to real authors, without newsroom review before publication (NBC News, 2025). Read against 3B's classification, this incident is a useful contrast case precisely because it sits clearly outside the legal serious-incident categories, no plausible harm to health, safety, critical infrastructure, fundamental rights, or the environment, even though it was reputationally damaging and widely covered. The correct classification of an incident like this is "operationally serious, legally not serious," and a response that spent scarce evidence-preservation and notification effort as if it met the legal bar would be misallocating exactly the discipline this topic teaches you to reserve for incidents that actually do.
Example 6 (a containment choice this program has already treated in depth, read for its throttle logic): the vendor indemnity conditions this program's contract topic covers. This program's treatment of AI vendor contracts shows that major vendors condition their liability protections on the deployer having its guardrails and content filters active. Read forward into this topic, that condition is itself a form of throttle: a vendor that expects its customers to keep filters on is implicitly telling every deployer which containment option preserves both safety and contractual protection at once. A deployer who disables its own safety filters to "simplify" a system, and only remembers the contract's guardrail condition after an incident, discovers the throttle it should have kept running was also the condition its own indemnity depended on, a reminder that 3C's containment menu and a vendor contract's fine print are not separate concerns.
Example 7 (a notification duty this program has already treated, read for its evidence angle): the conformity file's evidence bar. This program's Module 5 treatment of assembling a conformity file establishes that a defensible record needs specific, inspectable evidence rather than a general assurance of good practice (see Topic 5.6). Read backward into this topic, that bar is exactly what 3D's evidence checklist is built to satisfy in advance: a conformity file assembled without a preserved model version, a sample of the actual affected outputs, and a documented containment decision has a hole precisely where an examiner is most likely to press, and no amount of confident narrative fills a hole a preserved record was supposed to occupy.
Where people go wrong
- "We contained it, so the incident is basically over." Containment stops new harm; it answers none of the classify, preserve, and account questions this topic covers. An incident with a fast, clean rollback and no classification, no evidence preservation, and no notification check is not finished, it has only stopped getting worse operationally.
- "Our own severity scale already tells us how serious this is." Topic 3.5's operational scale answers "how urgently must we move," which is a different question from "does this meet a legal standard for serious," which is answered against the specific harm categories in 3B, not against how loud the incident is internally or on social media.
- "If it is not on the news, it is not serious." Reach and reputational visibility are not the legal test. A quiet incident affecting a handful of people can meet the serious-incident bar if the harm is to health, safety, fundamental rights, critical infrastructure, or the environment; a viral, embarrassing incident can miss the bar entirely if the actual delivered harm was reputational.
- "Disable everything, always, to be safe." Disabling a whole system when a narrow throttle would have stopped the harm is not automatically the safer choice; it is a choice with its own cost, denying a working feature to every uninvolved user, and it should be made deliberately against the menu in 3C, not by default.
- "We will figure out what to preserve after we understand what happened." Understanding what happened is exactly what the preserved evidence is for; waiting until you understand it to decide what to keep is how the evidence that would have let you understand it disappears first.
- "Cleaning up the bad outputs is part of fixing the problem." Deleting the harmful outputs from your own systems removes the single most direct proof of what the system actually did. Fix the system; do not delete the evidence of what it did before the fix.
- "We do not need an evidence owner; everyone knows to preserve the logs." Everyone knowing is how nobody does it. A named role, distinct from the incident commander, whose only job is confirming preservation, is what turns "we should keep that" into "someone confirmed we kept that."
- "The reporting clock starts once we are sure what happened." The Article 73 clock, like the GDPR clock, starts on awareness of a plausible serious incident, not on a completed investigation. Waiting for certainty before the clock "starts" in your own head is how a fifteen-day deadline gets missed while the investigation is still, technically, ongoing.
- "We checked whether the AI Act applies; that is our notification duty covered." Obligations stack. An incident can trigger an AI-specific report, a data-protection report, a contractual notice to an insurer, and an internal leadership duty, all at once, on different clocks. Checking one row and stopping is a common and costly incomplete answer.
- "We will report once and that is the deadline met." A preliminary report on incomplete facts, followed by a fuller report as the picture clarifies, is the normal and expected pattern under Article 73, not a failure to have "gotten it right the first time." Waiting for a complete report is what actually risks missing the clock.
- "Our system is not high-risk yet, so none of this applies to us." The Digital Omnibus deferral changes when a specific legal duty attaches, not whether the underlying discipline (classify, choose, preserve, account) is worth building now. An organization that waits for the deadline to attach before building the habit is improvising exactly when the stakes are highest.
- "The board just needs to know we handled it." A leadership briefing that offers reassurance instead of the classification, the containment logic, and the notification status gives a board nothing it can act on, and a board that later learns it was told less than the team knew has a trust problem layered on top of the incident.
- "We will write the leadership briefing once, at the end, when we have the full picture." The three-checkpoint discipline (hour one, hour eight, hour forty-eight) exists because leadership needs to make decisions during the window, not read a summary after it closes; a single end-of-window briefing means leadership was flying blind for the two days when decisions actually had to be made.
- "If we cannot classify it confidently, we should wait to report until we can." The honest exception in 3J is explicit: report provisionally where a duty's trigger is plausibly met, state the uncertainty, and keep preserving. Silence while you wait for certainty is not a neutral choice; it is a choice to let the clock run unanswered.
- "A throttle is a half-measure; only a full disable shows we take this seriously." A throttle that correctly narrows the harmful input pattern while preserving a working feature for uninvolved users is not a half-measure, it is frequently the more disciplined choice, because it is proportionate to what you actually know rather than performative.
- "New Zealand had no AI incident law, so Pak'nSave did nothing wrong." Not having a legal notification duty to break is not the same as having run a disciplined response. The four verbs, classify, choose, preserve, account, are worth doing even where no specific law compels the last one, because the first three protect real people and the organization's own ability to explain itself regardless of jurisdiction.
- "We already have an incident runbook from Topic 3.5; this is redundant." The Topic 3.5 runbook governs the first hour's operational response. This topic's playbook governs the classification, containment choice, evidence, and notification obligations across the following 47 hours, a genuinely different set of decisions that the first-hour runbook does not and should not try to also cover.
- "Once we file the AI-specific report, the incident is legally closed." Filing one applicable row of the notification matrix is not the same as completing it. A single incident routinely triggers more than one obligation at once, and each row runs on its own clock to its own recipient until it, too, is checked and actioned.
- "We operate in one country, so we only need to think about one regulator." Many AI systems serve users across borders even when the company itself is headquartered in one place. An incident affecting users in multiple jurisdictions can implicate more than one authority and more than one legal regime at once, and a playbook built around a single country's law will be blind to the others.
- "The evidence checklist is the legal team's job, not engineering's." The evidence lives in engineering's systems (model versions, logs, deployment history), and only engineering can execute the preservation steps in time; legal counsel interprets what the evidence means and what it obliges the organization to do, but cannot preserve data it does not have direct access to capture.
- "If leadership does not ask for an update, we do not need to send one." The three-checkpoint discipline is proactive by design. Waiting for leadership to ask is how a serious incident's first news to the board becomes a journalist's phone call rather than the team's own briefing, exactly the self-inflicted second crisis this topic exists to prevent.
- "A preliminary report locks us into that version of events." A preliminary report is explicitly framed as incomplete, naming what is known, what is suspected, and what remains under investigation; it is not a final position the organization is bound to defend unchanged, and a fuller report that updates the picture as facts clarify is the expected, normal follow-up, not a contradiction of the first filing.
- "We should decide who is at fault before we decide what to report." Assigning blame is not a prerequisite for classification or notification, and chasing it first delays both; the classification test asks what harm occurred or could plausibly occur, not who caused it, and the notification clock does not wait for an internal fault-finding process to conclude.
Questions people ask
- What is 48-hour incident playbook?
- The four-part document this topic's lab produces: a classification framework, a containment decision tree, an evidence-preservation checklist with a named owner, and a notification matrix, built in advance and filled in during a real incident.
- What is legal severity classification?
- The test of whether an incident meets a defined legal standard for "serious," independent of and sometimes in tension with an organization's own operational severity scale.
- What is serious incident (EU AI Act)?
- Under Regulation (EU) 2024/1689, an incident that directly or indirectly leads, or might lead, to the death of a person or serious harm to health, a serious and irreversible disruption of critical infrastructure, an infringement of Union law protecting fundamental rights, or serious harm to property or the environment. More on Serious incident (EU AI Act)
- What is containment menu?
- The three real options for stopping an AI incident's harm: rollback (return to a known-good version), throttle (narrow inputs, outputs, or reach without full shutdown), and disable (turn the feature off entirely), chosen deliberately against the cause and reach of the specific incident.
- What is throttle?
- A containment action that restricts what an AI system is allowed to do, its input format, its output type, or the population it serves, without shutting it down entirely; proportionate when the harm traces to a specific, identifiable pattern.
Keep going
This lesson builds AI incident response and recovery, and that page shows the roles that hire for it. Every Certified AI Governance Professional (CAIGP) lesson.
Watch the lesson and prove it
Proving it happens inside the program: watch, explain it in your own words, and it goes on a record employers can check.
Watch the lesson and prove it →