Calibration
The property that a system's stated confidence matches its real accuracy, so that instances it reports as 90 percent confident are correct about 90 percent of the time on your data. A confidence threshold is only a safe boundary if the confidence is calibrated; an uncalibrated model routes confident errors past the human.
Defined in 6 GAGE programs, which carry 13 distinct definitions of it. The wording above is taught in Certified AI Governance Professional (CAIGP).
How each discipline defines it
The same term does different work depending on who is using it. These are the definitions as each program teaches them, unedited.
The property of a system's confidence scores accurately reflecting its actual reliability, such that a stated ninety percent confidence corresponds to being correct roughly ninety percent of the time on that class of input. A well-calibrated system is the engineering target this topic argues for, rather than simply a maximally cautious one.
The property that a system's stated confidence matches its real accuracy, so that instances it reports as 90 percent confident are correct about 90 percent of the time on your data. A confidence threshold is only a safe boundary if the confidence is calibrated; an uncalibrated model routes confident errors past the human.
The degree to which a forecaster's stated confidence matches reality, so that of all things called 70 percent likely, about 70 percent occur. The defining trait of Tetlock's superforecasters and the real test of a forecaster, human or machine, above raw accuracy on any single call.
The property of a confidence score being honest: among predictions the model reports at a given confidence, it is correct at roughly that rate. An overconfident model (says 90, right 70) is dangerous because users trust predictions they should question.
The process of testing whether independent assessors using the same criteria produce the same rating, and rewriting the criteria where they do not. Its output is a changed document, not a trained audience.
Where it is taught
The exact lessons this term appears in. The first module of every program is free with a free account.
- AI-Suitable vs. Human-Essential Decision Trees · Critical Thinking and Context Engineering, Certified AI Practitioner: Workplace Foundations
- AI for Innovation: Predictive Analytics Applications · Advanced AI Literacy, Certified AI Practitioner: Workplace Foundations
- Self-Assessment: Comprehensive Knowledge Check · Assessment and Continuous Learning, Certified AI Practitioner: Workplace Foundations
- The trust boundary: what this system may decide alone and where a human signs · Evaluation and Trust, Certified AI Governance Professional (CAIGP)
- The evaluation report: evidence that your trust levels are earned, not hoped · Evaluation and Trust, Certified AI Governance Professional (CAIGP)
- The viva: defending your command live against the examiner · The Capstone: The Board Audit and Viva, Certified AI Governance Professional (CAIGP)
- Analytics, Telemetry and Observability · Technology, Platforms and Vendors, Certified AI Transformation Professional (CATP)
- Risk criteria · Enterprise Risk Architecture and Risk Appetite, Certified Enterprise Governance, Risk and Compliance Professional (CEGP)
- Bias, calibration, and confidence · Risk Identification, Scenarios, and Assessment, Certified Enterprise Governance, Risk and Compliance Professional (CEGP)
- Inherent risk, tiering, and due diligence · Third-Party, Supply-Chain, and Concentration Risk, Certified Enterprise Governance, Risk and Compliance Professional (CEGP)
Where it shows up at work
A definition is where Calibration starts. The lessons above teach it inside the competencies below, which are what job descriptions ask for and what the GAGE credential records. Every link here is drawn from the same careers map that builds a role's readiness check.
Knows when to trust, verify, escalate or override an AI recommendation, and stays accountable for the decision.
3 lessons on this page teach it as a primary outcome.
Defines who reviews AI outputs, what they check, when they can override, and how to keep review from becoming a rubber stamp.
2 lessons on this page teach it as a primary outcome.
Decides where AI should be used, where it should not, and which use cases create measurable value, tied to the organization's priorities.
One lesson on this page teach it as a primary outcome.
Roles that hire for it
Each role page shows the full competency set, what the seat decides, and a readiness check against your own evidence.
- Operations Manager, AI-readyneeds it at depth
- AI Product Manager, responsible product editionneeds it at depth · 3 open postings this week
- Ethical AI Specialistneeds working knowledge · 5 open postings this week
- AI Literacy Leadneeds working knowledge
- VP of AI Governance and Ethicsneeds working knowledge
- Autonomous Systems Governance Lead, robotics and physical AI editionneeds working knowledge
How job descriptions say it
Postings rarely use the textbook word. These are the phrases the careers map reads as asking for the competencies above.
Where the frameworks place it: EU AI Act Article 14; NIST AI RMF GOVERN; ISO/IEC 42001 Annex A.
One interview question about it
A model recommends a decision that affects a person. What do you check before you act on it, and when do you overrule it?
Asked of candidates for judgment when ai supports a decision. Answering it in your own words is exactly how a GAGE topic is passed, and the credential records the pass where an employer can check it.
If you come from another field
Some backgrounds already carry this competency, and the readiness check gives credit for it.
- Operations, claims, customer service and administration carries it as a claim to back up: Deciding when to escalate a case is judgment under uncertainty.
- A business, MBA or technology management degree carries it as a claim to back up: Strategy and opportunity triage are the core of the degree.
Terms it appears with
Not an alphabetical neighbourhood: these are the terms taught in the same lessons, ranked by how often they appear together.