Reproducibility
The property of a research finding that allows an independent party to run the same methodology, using shared code, data, or a described procedure, and check whether they get the same result. Reproducibility is what separates rung 3 (peer reviewed and independently checked) from rung 4 (a self-reported, unaudited claim), regardless of how technically sophisticated the claim sounds.
Defined in 4 GAGE programs, which carry 5 distinct definitions of it. The wording above is taught in Certified AI Data Governance Professional (CADGP).
How each discipline defines it
The same term does different work depending on who is using it. These are the definitions as each program teaches them, unedited.
The property of a research finding that allows an independent party to run the same methodology, using shared code, data, or a described procedure, and check whether they get the same result. Reproducibility is what separates rung 3 (peer reviewed and independently checked) from rung 4 (a self-reported, unaudited claim), regardless of how technically sophisticated the claim sounds.
Obtaining consistent results using the same input data, code, methods, and conditions of analysis as an original study, typically carried out by a different team or computing environment, per the National Academies of Sciences 2019 consensus framework. Confirms a released pipeline functions as described.
The property that a stranger with your golden set, your metric definition, and your grader can rerun the evaluation and obtain the same number. Reproducibility is the practical difference between a defensible result and an assertion a regulator can challenge.
Whether a competent person who was not present could regenerate the item from the source and get the same result. The property that survives hostile examination.
Whether a finding recurs reliably, intermittently, or once. It shapes the mitigation but never justifies dismissing a severe finding, because an attacker has unlimited free attempts and an intermittent exploit becomes a reliable one in their hands.
Where it is taught
The exact lessons this term appears in. The first module of every program is free with a free account.
- Reading the frontier honestly: which data-governance news changes your decisions · The Frontier Discipline, Certified AI Data Governance Professional (CADGP)
- Building the eval suite that would have caught your Module 3 incident · Evaluation and Trust, Certified AI Governance Professional (CAIGP)
- Red-teaming your own system: attacks a motivated user will find · Evaluation and Trust, Certified AI Governance Professional (CAIGP)
- Evidence engineering and governance, risk and compliance data architecture · Governance, Risk and Compliance Technology, Automation, and Evidence Engineering, Certified Enterprise Governance, Risk and Compliance Professional (CEGP)
- Reproducing a result and learning why it works · Research, Benchmarking, and Staying at the Frontier, Engineering Judgment and Professional Formation
Where it shows up at work
A definition is where Reproducibility starts. The lessons above teach it inside the competencies below, which are what job descriptions ask for and what the GAGE credential records. Every link here is drawn from the same careers map that builds a role's readiness check.
Designs tests for factuality, robustness, fairness, safety and abuse resistance with rubrics, baselines and thresholds, and says what a score misses.
One lesson on this page teach it as a primary outcome.
Knows when to trust, verify, escalate or override an AI recommendation, and stays accountable for the decision.
One lesson on this page teach it as a primary outcome.
Identifies value conflicts in an AI use, asks who benefits and who bears risk, and turns principles into criteria a review can apply.
One lesson on this page teach it as a primary outcome.
Roles that hire for it
Each role page shows the full competency set, what the seat decides, and a readiness check against your own evidence.
- Chief Technology Officer, AI engineering editionneeds it at depth
- Model Risk Managerneeds it at depth
- AI Evaluation Specialistneeds it at depth · 6 open postings this week
- AI Model Validatorneeds it at depth
- AI Product Manager, responsible product editionneeds it at depth · 3 open postings this week
- Ethical AI Specialistneeds it at depth · 5 open postings this week
How job descriptions say it
Postings rarely use the textbook word. These are the phrases the careers map reads as asking for the competencies above.
Where the frameworks place it: NIST AI RMF MEASURE 1 and 2; NIST AI 600-1; EU AI Act Article 14; NIST AI RMF GOVERN.
One interview question about it
Design the evaluation for a customer-service model before launch. What do you test, against what data, and what result blocks the release?
Asked of candidates for ai evaluation and testing design. Answering it in your own words is exactly how a GAGE topic is passed, and the credential records the pass where an employer can check it.
If you come from another field
Some backgrounds already carry this competency, and the readiness check gives credit for it.
- Model risk, credit risk and quantitative analysis carries it as work you can show: Test design and benchmark selection are the craft.
- A computer science or software engineering degree carries it as a claim to back up: Testing is second nature; AI evaluation design is a new shape of it.
- A data science, statistics or analytics degree carries it as a claim to back up: Evaluation design for a governance decision is a new frame for a known skill.
Terms it appears with
Not an alphabetical neighbourhood: these are the terms taught in the same lessons, ranked by how often they appear together.