Reproducibility
The property of a research finding that allows an independent party to run the same methodology, using shared code, data, or a described procedure, and check whether they get the same result. Reproducibility is what separates rung 3 (peer reviewed and independently checked) from rung 4 (a self-reported, unaudited claim), regardless of how technically sophisticated the claim sounds.
Defined in 3 GAGE programs, which carry 4 distinct definitions of it. The wording above is taught in AI Data Governance: The Data Chair.
How each discipline defines it
The same term does different work depending on who is using it. These are the definitions as each program teaches them, unedited.
The property of a research finding that allows an independent party to run the same methodology, using shared code, data, or a described procedure, and check whether they get the same result. Reproducibility is what separates rung 3 (peer reviewed and independently checked) from rung 4 (a self-reported, unaudited claim), regardless of how technically sophisticated the claim sounds.
Obtaining consistent results using the same input data, code, methods, and conditions of analysis as an original study, typically carried out by a different team or computing environment, per the National Academies of Sciences 2019 consensus framework. Confirms a released pipeline functions as described.
The property that a stranger with your golden set, your metric definition, and your grader can rerun the evaluation and obtain the same number. Reproducibility is the practical difference between a defensible result and an assertion a regulator can challenge.
Whether a finding recurs reliably, intermittently, or once. It shapes the mitigation but never justifies dismissing a severe finding, because an attacker has unlimited free attempts and an intermittent exploit becomes a reliable one in their hands.
Where it is taught
The exact lessons this term appears in. The first 7 topics of every program are free with a free account.
- Reading the frontier honestly: which data-governance news changes your decisions · The Frontier Discipline, AI Data Governance: The Data Chair
- Building the eval suite that would have caught your Module 3 incident · Evaluation and Trust, AI Governance: Applied Mastery
- Red-teaming your own system: attacks a motivated user will find · Evaluation and Trust, AI Governance: Applied Mastery
Terms it appears with
Not an alphabetical neighbourhood: these are the terms taught in the same lessons, ranked by how often they appear together.