Designing a Control That Can Be Tested
The short answer
A control has to be able to fail, or it is not a control
The dividing line between a real control and a policy statement or a good intention is whether anyone can say, in advance, what failure would look like and who would notice. Evolv's marketing claims about weapons detection accuracy had no stated way to fail, which is precisely what the FTC's complaint turned on.
What you will be able to do
- Distinguish a control from a policy statement and from a good intention, using the presence or absence of a stated way to fail as the dividing line, so you can tell in one read whether a sentence in your governance file is a control or only the sound of one.
- Write a control in fully testable form, naming its owner, its trigger, its frequency, the evidence artifact it produces every time it runs, and a test procedure that states a sample size and a pass condition.
- Design a test procedure for any control, choosing a sample size an auditor would accept as representative and a pass condition specific enough that two different reviewers would reach the same verdict from the same evidence.
- Build three kinds of control, an intake control, a monitoring control, and an access control, for one real AI system, each written so a stranger could run the test without asking you what it means.
- Analyze a real enforcement failure, the FTC's action against Evolv Technologies, to see exactly what a testable control would have looked like where an unsupported performance claim stood in its place.
- Rewrite a policy sounding control so that it can fail a test, and state precisely what evidence a real reviewer would need to sample to run that test, the exact move this topic's teach back interrogates.
- Defend a control under challenge, answering the questions a hostile auditor asks about any control: who owns it, when does it run, what does it produce, and what result would count as a failure.
The ideas, one by one
Five elements or it is not testable: owner, trigger, frequency, evidence artifact, test procedure
Each answers a question an auditor will otherwise ask you live. A control missing any one of them is not a weaker control; it is a different, unverifiable kind of object.
The test procedure needs two specific parts: a sample size and a pass condition
A sample size vague enough to mean "a few outputs" and a pass condition vague enough to mean "results look reasonable" together guarantee that two different reviewers, testing the same control, would reach two different verdicts, which defeats the entire purpose of testing.
Set the pass condition before you see the results
A threshold decided after looking at the evidence is not a test; it is a story fit to whatever the data happened to show. The whole value of stating it in advance is removing the reviewer's discretion to call a bad result acceptable.
A policy statement and a good intention both masquerade as controls, and both are more dangerous than an honest gap
They occupy the slot where a real control belongs and make the absence of governance invisible, right up until an auditor, or a regulator, asks to see the test.
Match the sample size and frequency to real stakes and real rates of change
A sample too small to catch a real failure rate proves nothing by chance; a frequency slower than the system's own rate of change means the control is testing a version of the system that no longer exists.
Three control types cover the moments a system actually passes through
An intake control watches what is let in, a monitoring control watches the system while it runs, and an access control watches who can touch it; a system with genuine, testable versions of all three has covered where governance failures most often live.
An unbroken record of passes is a reason to look harder, not a trophy
A control that has never once failed is at least as likely to be testing something trivial or carrying a pass condition too loose to ever fail, as it is to be genuinely robust; treat a spotless record with the same scrutiny you would give a suspicious one.
Design the test so a stranger could run it, because eventually one will
The Module 13 board inspection and viva, and any real auditor or regulator, will ask to execute your test procedure on a sample they choose, not the one you would have picked; a control built to that standard survives, and one that was only ever a sentence about a control does not.
Operating and being correct are two different claims, and only a test procedure proves the second one
A control can run flawlessly on schedule and produce its evidence artifact every time while quietly being wrong, and only an independent test that recomputes the underlying evidence, not merely confirms the artifact arrived, can catch that gap.
A failed test with no remediation is a diagnosis with no treatment
Recording that a control failed changes nothing about the underlying risk unless it is followed by a named owner for the fix, a concrete plan, and a re test date confirming the fix actually worked; without those three, a failed test and no test at all leave the organization in the same position, just one with more paperwork.
Listen to it
The same lesson, talked through by two hosts as episode 82 of the podcast.
Real cases
These are documented cases used to illustrate testable control design, not to predict your organization. Each is cited and used for a specific point.
Example 1: FTC v. Evolv Technologies, the claim with no test behind it (this topic's anchor). The FTC alleged that Evolv Technologies marketed its Evolv Express AI weapons detection system as detecting all weapons and ignoring harmless items more accurately than a metal detector, while the system failed in several documented instances, including missing a seven inch knife in October 2022 that was later used to stab a student. The company settled, under a stipulated order approved 11 March 2025, agreeing not to make unsupported capability claims and accepting ongoing compliance reporting. (FTC press release, 26 November 2024; FTC, Stipulated Order, entered 11 March 2025.) The value for this topic is precise: the gap between Evolv's claim and a real control was exactly the five elements in Section 3C, an owner, a trigger, a frequency, an evidence artifact, and a test procedure with a sample size and a pass condition, all absent from a sentence that otherwise read exactly like a governance commitment. A separate civil suit brought by the stabbed student against Evolv, its installer, and the school district goes further than the FTC's complaint: it alleges Evolv "falsified and doctored" the test results it used to support its detection claims (Louisville Public Media, "N.Y. student sues company that makes JCPS' new weapons detectors, saying knife got through scanner," 21 November 2023). If proven, that allegation is not merely a missing test procedure, it is Section 3G's fourth failure pattern, gaming the test, at its most direct: a sample and a result chosen to produce the answer already wanted, not to find out whether the system actually worked.
Example 2: Knight Capital and a rollback control that existed on paper only. Knight Capital's 2012 trading system disaster traced to old, disabled code that reactivated on deployment because the deployment process lacked a genuinely tested verification step confirming every server ran the correct, current code before going live (see Topic 3.5). Owned by Topic 3.5, referenced here only for the control lesson: a deployment verification "control" that is never actually tested against a real, adversarial sample of servers is functionally no control at all, a lesson this topic's intake control category is built to prevent.
Example 3: The EU AI Act's post market monitoring duty assumes testable controls exist. Article 72 of Regulation (EU) 2024/1689 requires providers of high risk AI systems to actively and systematically collect and analyze performance data after deployment, an obligation that is meaningless without a defined sample, a defined frequency, and a defined threshold for what counts as a problem (Regulation (EU) 2024/1689, Article 72; established). The Digital Omnibus on AI deferred when that duty binds a stand alone Annex III system to 2 December 2027 (Regulation (EU) 2026/1744; established), so the deadline has not yet passed for most providers; the example still matters because it shows a binding legal duty, not merely good practice, presumes the five element structure this topic teaches, and a deferred deadline is a reason to build the control now, unrushed, rather than a reason to skip it.
Example 4: ISO/IEC 42001's internal audit clause requires a checkable standard. The certifiable AI management system standard requires an internal audit function that evaluates whether the organization's stated controls are actually effective, which is impossible to do meaningfully against a policy statement with no stated pass condition (ISO/IEC 42001:2023; established, voluntary standard). The example shows that even a management systems standard, not a specific incident, drives the same requirement: a control that cannot be tested cannot be audited, certified, or trusted.
Example 5: The generic pattern behind failed compliance inspections. Across regulated industries, a recurring finding in failed inspections is not that no control existed but that the control on paper was never actually run, tested, or evidenced the way its description implied, so the inspector's real finding was the gap between the documented control and the operating one (a pattern visible across financial, safety, and data protection audit findings generally). The point for this topic is structural: the design stage of the control lifecycle is usually fine; the operate and test stages are where real programs quietly fail, which is why 3H treats all four lifecycle stages as equally necessary.
Example 6: A vague pass condition that could never fail. A pattern noted across public reporting on AI vendor accuracy claims, including the allegations in the Evolv matter, is language such as results being "generally accurate" or "highly reliable" with no stated threshold, sample, or comparison baseline (FTC, Complaint, 26 November 2024, describing Evolv's marketing language pattern). The point for this topic: a pass condition without a number or a specific checkable state is not a pass condition, it is marketing language wearing a lab coat, and the fix is always to force a number or a specific state before accepting the sentence as a control.
Example 7: The right to cancel as a remedy for an unproven claim. Under the FTC's stipulated order, certain K to 12 school customers who signed contracts with Evolv between April 2022 and June 2023 gained the right to cancel those contracts (FTC, Stipulated Order, entered 11 March 2025). The point for this topic is about remediation, the fourth stage of the control lifecycle in 3H: when a claim turns out to have been unsupported all along, a real remedy for the people who relied on it, not merely a promise to do better going forward, is part of what closing the gap actually requires, and it is a useful reminder that a control failure discovered late can carry a cost measured in more than an audit finding.
Example 8: A management standard's evidence expectation, applied to controls specifically. ISO/IEC 42001:2023 expects an organization to maintain documented information and make evidence of its controls available for audit, language that presumes controls exist as verifiable objects, not narrative descriptions (ISO/IEC 42001:2023; established, voluntary standard). The point for this topic, distinct from Example 4's audit function point: the standard's baseline expectation is that a control's evidence can be produced on request at all, which is the same minimum bar this topic's evidence artifact element sets, before any question of testing frequency or sample size even arises.
Where people go wrong
- Treating a policy statement as if it were already a control. "Flagged decisions are reviewed by a human" sounds like a control but names no owner, trigger, frequency, evidence artifact, or pass condition. Test every sentence in your governance file against the five elements in 3C before trusting it as a control.
- Writing a pass condition after looking at the results. Deciding whether the evidence "looks fine" once you have already seen it is not a test; it is a story fit to the data. Commit to the sample size and pass condition before running the test, exactly as the Evolv case shows a company never doing for its own accuracy claims.
- Naming a team or department as the owner instead of a role. "The compliance team" is not accountable for anything specific; when the control lapses, no one person answers for it. Name the exact role, the way this program names an accountable maker on every annex entry (see Topic 10.6).
- Choosing a sample size too small to reveal a real failure rate. Reviewing two or three outputs and calling the control verified tells you almost nothing about the rest of the population. Match the sample size to the volume and the stakes, and prefer testing the whole population when it is small enough to do so cheaply.
- Writing a pass condition so loose that nothing could fail it. "At least one output was reviewed" or "results were generally acceptable" will always pass and prove nothing. A defensible pass condition states a specific threshold or state precisely enough that two reviewers would reach the same verdict independently.
- Testing a control only once, at launch, and never again. A control verified before deployment and never revisited is a control that has already expired by the time the system changes underneath it. Set a frequency and tie it to the events that actually change the system's risk, not to a one time launch checklist.
- Assuming the evidence artifact exists because the control's description says it does. A control that claims to generate a monthly report is worthless if the report generation was quietly disabled or never fully built. The test procedure must confirm the artifact is actually produced and dated, not merely trust the description.
- Letting the control's usual owner also design and run its own test. A test designed and executed entirely by the person who benefits from the control looking good is prone to gaming, consciously or not. Have someone other than the control's day to day owner run the periodic test wherever possible.
- Treating an unbroken record of passes as proof the control is strong. A control that has never once failed is at least as likely to be testing something trivial, or to have a pass condition set too loosely, as it is to be genuinely robust. Treat a spotless record as a prompt to re examine the test's rigor, not as a trophy.
- Skipping remediation after a failed test. A control that fails and is followed by nothing is the same as a control that was never tested, with an extra layer of documented denial on top. Every failed test needs a named owner for the fix and a re test date.
- Confusing a real activity with a testable control. Adjusters really reviewing claims, or engineers really checking a model, is not the same as having a testable control around that activity. The activity can be genuine while the way of verifying it happened, on schedule, to a stated standard, does not yet exist; build the verification, not just the activity.
- Believing marketing language about a product's performance counts as an internal control over that product. A vendor's or your own team's confident description of what a system does is not evidence that anyone tests it against a stated sample and pass condition on an ongoing basis, which is exactly the gap the FTC's Evolv complaint turned on.
- Assuming four strong elements mean the fifth is unnecessary. A control with a real owner, trigger, frequency, and evidence artifact still cannot be verified by anyone but its author until it also carries a test procedure with a sample size and a pass condition; a control this close to complete is exactly the kind reviewers tend to wave through without checking for the missing piece.
- Treating remediation as optional once a test is logged. Recording a failed test without naming a fix owner and a re test date leaves the underlying risk exactly where it was, only now with documented proof the organization knew. A logged failure with no remediation plan is a diagnosis with no treatment.
Questions people ask
- What is control (testable)?
- A defined activity, performed by a named owner, triggered by a specific event or schedule, that produces evidence every time it runs and can be tested against a stated pass condition using a stated sample. The dividing test: can anyone say, in advance, what would have to happen for it to fail, and who would notice.
- What is owner (of a control)?
- The specific, named role, not a team or a department, accountable for a control actually running and for the evidence it produces. A control with no named owner has, in practice, no one accountable when it lapses.
- What is trigger?
- The exact event or schedule that starts a control, stated as either time based (a date or interval) or event based (a specific occurrence such as a retraining or a flagged decision). A control with no trigger runs only when someone happens to remember it. More on Trigger
- What is frequency?
- How often a control repeats, stated as a number or an interval, separating a genuine, ongoing control from a one time check performed at launch and never revisited.
- What is evidence artifact?
- The specific, named thing a control produces every time it runs, such as a signed review log, a sampled output report, or an access review record. The thing an evidence annex entry ultimately points to (see Topic 10.6).
Keep going
This lesson builds AI evaluation and testing design, and that page shows the roles that hire for it. Every Certified AI Governance Professional (CAIGP) lesson.
Watch the lesson and prove it
Proving it happens inside the program: watch, explain it in your own words, and it goes on a record employers can check.
Watch the lesson and prove it →