Benchmark
A standardized test or task used to measure and compare AI model performance on a specific, defined capability, such as a set of exam questions or coding problems. A strong benchmark result demonstrates performance on that specific, bounded task; it does not, by itself, establish general real-world capability, as Section 3J's worked example demonstrates.
Defined in 2 GAGE programs, which carry 5 distinct definitions of it. The wording above is taught in The AI Lobbyist: Certified AI Policy Strategist.
How each discipline defines it
The same term does different work depending on who is using it. These are the definitions as each program teaches them, unedited.
A standardized test or task used to measure and compare AI model performance on a specific, defined capability, such as a set of exam questions or coding problems. A strong benchmark result demonstrates performance on that specific, bounded task; it does not, by itself, establish general real-world capability, as Section 3J's worked example demonstrates.
A shared, public test set used to compare models against each other in general. Useful for ranking, almost never containing your domain, users, or incident, and sometimes compromised by who funded it or who saw the questions, so it is not a substitute for your own suite.
A standardized test used to score and compare systems. It is evidence only once you can answer whose test it is, who chose or funded it, and whether the scored system is the one you would deploy; otherwise it is a number that functions like a demo.
A fixed set of tasks with known answers used to measure and compare AI models. A benchmark score is the outcome of running a model against those tasks under a specific procedure, not an intrinsic property of the model.
A standardized test used to measure an AI model's performance on a specific, defined task type, such as MMLU for academic multiple-choice questions; a benchmark score measures a narrow capability, not general intelligence.
Where it is taught
The exact lessons this term appears in. The first module of every program is free with a free account.
- Distrust is a skill: why demos convince and evals do not lie · Evaluation and Trust, Certified AI Governance Professional (CAIGP)
- Building the eval suite that would have caught your Module 3 incident · Evaluation and Trust, Certified AI Governance Professional (CAIGP)
- Reproducing a claim: testing a vendor benchmark yourself in an afternoon · Staying Current: The Frontier Discipline, Certified AI Governance Professional (CAIGP)
- What AI Really Is: No Hype, No Myths, No Fear · Foundations of AI for Lobbyists, The AI Lobbyist: Certified AI Policy Strategist
- How AI Actually Works Under the Hood: Architectures and Limits for Policy People · Technical Credibility Deep Dive, The AI Lobbyist: Certified AI Policy Strategist
Where it shows up at work
A definition is where Benchmark starts. The lessons above teach it inside the competencies below, which are what job descriptions ask for and what the GAGE credential records. Every link here is drawn from the same careers map that builds a role's readiness check.
Explains training, tokens, context windows, embeddings, retrieval and fine-tuning well enough to ask an engineer a precise question and spot weak evidence.
One lesson on this page teach it as a primary outcome.
Decides what a person affected by an AI decision must be told, how an output can be explained, and how they can challenge it.
One lesson on this page teach it as a primary outcome.
Designs tests for factuality, robustness, fairness, safety and abuse resistance with rubrics, baselines and thresholds, and says what a score misses.
One lesson on this page teach it as a primary outcome.
Roles that hire for it
Each role page shows the full competency set, what the seat decides, and a readiness check against your own evidence.
- AI Model Validatorneeds it at depth
- Chief Technology Officer, AI engineering editionneeds it at depth
- AI Evaluation Specialistneeds it at depth · 6 open postings this week
- AI Product Manager, responsible product editionneeds it at depth · 3 open postings this week
- AI Security Architectneeds it at depth · 8 open postings this week
- AI Governance Engineerneeds it at depth · 7 open postings this week
How job descriptions say it
Postings rarely use the textbook word. These are the phrases the careers map reads as asking for the competencies above.
Where the frameworks place it: EU AI Act Articles 13 and 50; NIST AI RMF trustworthiness characteristics; NIST AI RMF MEASURE 1 and 2; NIST AI 600-1.
One interview question about it
Explain how a large language model produces an answer, at the depth a governance decision needs and no deeper.
Asked of candidates for how models work, at a governance depth. Answering it in your own words is exactly how a GAGE topic is passed, and the credential records the pass where an employer can check it.
If you come from another field
Some backgrounds already carry this competency, and the readiness check gives credit for it.
- Data analysis, stewardship and business intelligence carries it as a claim to back up: SQL and analytical fluency make the technical layer approachable.
- Model risk, credit risk and quantitative analysis carries it as work you can show: Quantitative fluency is demonstrated in your work.
- A computer science or software engineering degree carries it as work you can show: You have trained or integrated a model in coursework or a project.
Terms it appears with
Not an alphabetical neighbourhood: these are the terms taught in the same lessons, ranked by how often they appear together.