Skip to main content

Governance that survives: rebuilding the file so the next attack finds less

The short answer

A patch closes the hole; a rebuild closes the reason the hole existed

The test is whether the fix holds when the attacker changes one step. Replika patched the signup gate, left the root cause (age as a form field), and lost five million euros when the varied attack still worked.

What you will be able to do

  • Distinguish a root-cause remediation from a surface patch, and predict which findings will reopen under a second attack.
  • Evaluate each attack finding against your rebuilt conformity file and judge whether the rebuild actually closes the weakness or merely relocates it.
  • Rebuild a governance claim so that it is grounded in evidence you can produce on demand, or cut it if you cannot defend it.
  • Shrink the attack surface of a file by removing unfalsifiable claims, unsupported numbers, and promises you cannot keep, so the next attacker has fewer places to push.
  • Prove a remediation worked by re-running the original attack against the rebuilt file and confirming it now fails (a regression proof), rather than asserting the fix in prose.
  • Assemble a remediation log that maps every finding to its root cause, its fix, and its re-test result, so the rebuild is auditable and not just claimed.
  • Justify which of two candidate rebuilds better survives the next attack, defending the choice against a hostile examiner.
  • Prefer a structural control that enforces an outcome over a promised control that depends on a person acting, and convert one into the other.
  • State what a rebuilt file honestly cannot claim, and mark forward-looking mitigations as established, emerging, or speculative rather than asserting them as done.

The ideas, one by one

A governance file has an attack surface: every claim you cannot defend

Unfalsifiable words, unsupported numbers, and unkept promises are targets you painted on your own file. Rebuilding usually means the file gets shorter, because you cut what you cannot defend.

Root cause governs a category; symptom governs an instance

Run the why-chain until the answer names a mechanism. Fixing the mechanism closes every finding that shares it, which is why the rebuild is more work now and far less work at the next attack.

Assert nothing; prove it with a regression re-run

Re-run the exact attack against the rebuilt file and show it now fails, with evidence. A demonstrated failure survives a skeptic; a sentence that says "we fixed it" is just more attack surface.

The residual-risk column earns the trust

A file that claims everything is closed reads as dishonest or naive. An honest open item with an interim control and a close date is the strongest trust signal in the whole file.

Sequence by harm and by shared root, not by report order

Fix first the high-harm findings and the root causes that close many holes at once. Spending the rebuild budget on cosmetics while a person-harming hole stays open is what regulators punish.

A rebuild with no trace reads as a cover-up

The remediation log turns a quiet fix into an auditable one and feeds the evidence annex in Topic 10.6 and the board inspection in Topic 13.2. Fix in a way you can show.

Some files cannot be rebuilt, and recognizing that is a decision, not a failure

When the underlying practice is indefensible, stopping it or conceding beats defending the indefensible in public. (see Topic 11.2)

The goal is not to win the last attack but to shrink the next one

Governance that survived one attack is not strong. Governance that gets smaller and harder each time it is attacked is.

Judge a rebuild by falsifiability and structure, not by confidence

The rebuild that reads as more thorough is often the weaker one, because thoroughness usually means more claims and more universal language. Read past confidence to whether each claim points at evidence and each control holds structurally.

Prefer structural controls to promised ones

A control that holds because of how the system works cannot silently lapse; a control that depends on someone remembering to act will, eventually, not be acted on, and the gap becomes the next finding. Rebuild toward controls that enforce themselves.

A rebuild is judged on whether it works, not on whether it was attempted

The Garante assessed Replika's controls both before and after the suspension and found both insufficient. Sincerity and effort earn nothing; a re-run that shows the attack failing earns everything. Prove, do not describe.

Deletion is a first-class rebuild move

Removing an undefendable, unneeded claim, field, or feature closes the finding and shrinks the surface at once, and leaves nothing to defend on that point. "What did we remove" is often a stronger answer to a regulator than "what did we add."

The rebuild never finishes

Treat it as a cadence: re-run stored attacks on a schedule, add each new capability to the attack list, keep the residual-risk log current. A file rebuilt once and frozen grows more attackable every month; a file rebuilt on a cadence stays small and hard.

Listen to it

The same lesson, talked through by two hosts as episode 89 of the podcast.

Real cases

These examples show the rebuild-versus-patch distinction in documented cases. The anchor case is treated in depth; the others are referenced to show the pattern travels.

Example 1 (anchor): Replika / Luka and the Italian Garante. On 2 February 2023 the Garante ordered Luka to stop processing Italian users' data through Replika, citing no valid legal basis and no age verification for a service that claimed to exclude minors (TechCrunch, "Replika, a virtual friendship AI chatbot, hit with data ban in Italy over child safety," 2023). Luka rebuilt: it added age verification and revised its privacy policy. But the follow-up technical assessment found the rebuild was a patch. A registered user could change their birth date to an adult date with no re-verification, and the cooling-off screen could be bypassed by using incognito browsing, because the control lived in the browser session rather than the account. The Garante concluded that the measures both before and after the suspension were insufficient, and on that basis fined Luka five million euros in May 2025 for breaching several articles of the EU General Data Protection Regulation (GDPR), the bloc's data-protection law, on legal basis, transparency, and data-protection-by-design (European Data Protection Board, "AI: the Italian SA fines company behind chatbot Replika," 2025; PwC Italy TLS analysis, 2025). The lesson is exact: the rebuild closed the visible gate and left the root cause, that age was a form field and not a governed account property, untouched, so the next assessment found more than the first.

Three details of the Replika resolution sharpen the lesson for your own rebuild. First, the regulator explicitly assessed the controls both before and after the suspension and found both insufficient, which is the clearest possible statement that a rebuild is judged on whether it works, not on whether it was attempted. Second, the fine cited data-protection-by-design and by-default among the breached obligations (the principle that protections are built into how a system works, reflected in Article 25 of the EU General Data Protection Regulation), which is exactly the difference between a structural control and a bolted-on gate. Third, alongside the fine the authority opened a separate inquiry into how the underlying generative model was trained, a reminder that closing one front can open another and that the rebuild is a discipline, not a finish line. A learner reading this file should take away not "Replika was careless" but "a fast, sincere, incomplete rebuild is the normal failure, and only root-cause remediation proven by re-test avoids it."

Example 2: A filing found hollow, then not rebuilt. In the MyPillow-related litigation a court sanctioned lawyers whose brief carried dozens of AI-hallucinated citations (owned in depth at (see Topic 11.1)). The instructive contrast for this topic is what a rebuild would have required: not deleting the bad citations, which is a patch, but changing how filings are assembled so no citation reaches a court unverified, which is a root-cause rebuild. Deleting the bad cites leaves the process that produced them intact, and the process produces more.

Example 3: Conceding rather than rebuilding. The Uber Eats courier case (owned at (see Topic 11.2)) shows the alternative to a rebuild: when the file cannot be rebuilt to survive, the defensible move is to concede and settle rather than defend an indefensible position to a full hearing. Rebuild is not always available. Recognizing that a file cannot be made to survive, and folding early, is itself a governance decision, and a cheaper one than a rebuild that fails in public.

Example 4: Re-attacking to size the rebuild. The PredPol re-run (owned at (see Topic 11.3)) is the diagnostic that should precede a rebuild: researchers re-ran the model on its own data and showed the feedback loop sending police back to the same neighborhoods. Before you rebuild, you re-attack to learn how deep the problem goes, so you fix the mechanism and not the symptom. A rebuild that skips the re-attack fixes what you assume is broken, not what is actually broken.

Example 5: The forensic reconstruction a weak rebuild invites. The Pikesville deepfake case (owned at (see Topic 11.4)) shows how investigators reconstruct the truth from digital records when accounts conflict. A rebuild that leaves a gap between what the file promises and what the logs show is a rebuild that invites exactly this reconstruction. The remediation log exists so that your own records tell a consistent story before anyone else reconstructs it for you.

Example 6: A rebuild that changed the mechanism (Brazil, cross-border data). When Brazil's data protection authority, the ANPD, ordered Meta to stop training AI on Brazilians' personal data in 2024 (owned in depth at (see Topic 6.6)), the durable response available to a company in that position is not a statement of intent but a change to the governing mechanism: alter what data may lawfully feed training and prove the change with records, rather than promise better behavior. The contrast with a patch is exact. A patch issues a press release and quietly narrows one data flow; a rebuild changes the lawful basis on which any training data is admitted and can show, on demand, which categories are now excluded and why. Regulators in different jurisdictions reward the same demonstrable mechanism change, which is why the rebuild move travels across borders even when the specific law does not.

Example 7: The pattern outside the West (South Korea, consent). The South Korean regulator's action against a chatbot built on messages collected for a different purpose (the consent-archaeology case owned at (see Topic 2.2)) is another instance of the same lesson from a different legal tradition. A company told to fix an unlawful reuse of data can patch by deleting the one dataset named, or rebuild by changing how any data is admitted to training so that provenance and consent are checked at the source. The patch invites the regulator to find the next dataset with the same defect; the rebuild closes the class. Across the OAIC in Australia, the CNIL in France, the PIPC in South Korea, and the Garante in Italy, the enforcement posture converges on rewarding the mechanism change proven by evidence, which is the whole thesis of this topic stated four ways.

Example 8: The rebuild that was smaller, not bigger. A recurring pattern in enforcement resolutions is that the organizations regulators treat most favorably are often the ones that removed a capability rather than adding controls around it. When a data practice cannot be made lawful, the durable fix is to stop it: a company that halts a data collection, retires a feature, or narrows a model's scope removes the finding and shrinks its exposure at the same time, and has nothing left to defend on that point. This is the deletion move from Section 3G-i seen in the wild. The lesson for the learner's own rebuild is that "what did we remove" is a legitimate and often stronger answer to a regulator than "what did we add," because every addition is a new promise and every removal is one fewer thing to break.

Where people go wrong

  • "We closed the finding, so we are safe." Closing the specific finding is a patch. Unless you closed the root cause, the next variant of the same attack still works. Replika closed the signup gate and lost anyway because the root cause, age as a form field, was untouched. Ask of every fix: does it hold if the attacker changes one step?
  • "A bigger, more detailed file is a stronger file." Usually the opposite. Every added claim is a new thing an attacker can test. Files that survive are typically shorter than the files that got attacked, because the rebuild strips out everything that cannot be defended. Adding reassurance adds attack surface.
  • "Assert the fix in the file and move on." An asserted fix is an unbacked claim, which is attack surface. Show the fix by re-running the original attack and demonstrating it now fails. A regression proof survives a skeptic; a sentence saying "we fixed it" does not.
  • "An empty residual-risk section shows we did a thorough job." To an experienced examiner it shows the opposite: either you did not look hard, or you are hiding something, because real rebuilds have limits. An honest residual-risk entry with an interim control and a close date is the strongest trust signal in the file.
  • "Fix the findings in the order they were reported." Report order is arbitrary. Sequence by harm and by shared root cause: fix first the high-harm findings and the root causes that close many holes at once. Spending the rebuild budget on a cosmetic finding while a person-harming hole stays open is a defensible-looking mistake that regulators punish.
  • "Delete the bad thing and the problem is gone." Deleting the bad citation, the bad data field, the bad output leaves the process that produced it intact, and the process produces more. Remediation targets the mechanism, not the instance.
  • "The rebuild is done when the file reads well." The rebuild is done when the old attacks re-run and fail, the changes are logged with evidence, the residual risk is stated, and an independent person could not break it with the original script. Reading well is not evidence.
  • "We can rebuild anything if we try hard enough." Some files cannot be made to survive, because the underlying practice is indefensible. Recognizing that, and conceding or stopping the practice rather than defending the indefensible, is a governance decision, not a failure. (see Topic 11.2)
  • "A private, quiet fix is fine as long as it works." A rebuild with no trace reads as a cover-up to a regulator or board. The remediation log is what turns a quiet fix into an auditable one. Fix in a way you can show.
  • "The most thorough-looking rebuild is the strongest." Often the reverse. A confident, detailed rebuild full of universal claims ("all," "fully compliant") and new policy documents carries more attack surface than a smaller rebuild whose claims point at logs and whose controls are structural. Judge a rebuild by the falsifiability of its claims and the structure of its controls, not by how impressive it reads.
  • "A policy document proves the control exists." A policy is a promise about behavior, not evidence that the behavior happens. The next forensic pass compares the policy to the logs, and if they disagree, the policy becomes evidence against you. A structural control (release blocked until a reviewer acts) proves itself through its own record; a policy does not.
  • "A sourced number that is precise looks stronger than a rounded one." Not always. A figure stated to a decimal it cannot actually support ("99.4 percent accurate") invites the exact question "measured how, on what set, with what margin," and an artifact that cannot answer that question at that precision has just handed the attacker a gap the rounded, honestly bounded version would not have. State a number no more precisely than your evidence actually supports; false precision is a subtler cousin of the unsupported number, and it is just as attackable.
  • "If the developers confirmed the fix, the file can state it as fixed." Developer confirmation is an assertion, and assertions are attack surface. Only a re-run of the original attack, showing it now fails with captured evidence, converts "fixed" into proof a skeptic can reproduce.

Questions people ask

What is conformity file?
The folder of evidence showing that a specific AI system meets the rules that apply to it: what the system is, what data it uses, how it was tested, who signs off, and what happens when it fails. First assembled in Topic 5.6; attacked across Module 11; rebuilt here; inspected in Topic 13.2. More on Conformity file
What is patch?
A fix that closes the exact path an attacker used without closing the reason the weakness existed. A patch survives the specific attack already lost and fails when the attacker varies one step.
What is rebuild?
A fix that closes the root cause of a weakness, so the whole family of attacks sharing that cause fails, not just the one reported path. A rebuild costs more now and far less at the next attack.
What is attack surface (of a file)?
The set of claims in a governance file that an attacker can challenge and the owner cannot defend. Made up mostly of unfalsifiable claims, unsupported numbers, and promises the practice does not keep. Rebuilding usually shrinks it.
What is unfalsifiable claim?
A statement that cannot be proven or disproven because it is not measurable ("robust," "state-of-the-art," "fully compliant"). It is pure exposure, because there is no fact to point an attacker at. Cut it or replace it with a measured statement.

Keep going

Watch the lesson and prove it

Proving it happens inside the program: watch, explain it in your own words, and it goes on a record employers can check.

Watch the lesson and prove it →