Effect size
A measure of how large a reported difference actually is, as distinct from whether it is statistically detectable at all; used in a synthesis matrix alongside a reported mean to judge whether a headline comparison between two methods reflects a meaningful practical gap or one that is technically present but small enough not to drive a real engineering decision on its own.
Defined in 2 GAGE programs, which carry 3 distinct definitions of it. The wording above is taught in Engineering Judgment and Professional Formation.
How each discipline defines it
The same term does different work depending on who is using it. These are the definitions as each program teaches them, unedited.
A measure of how large a reported difference actually is, as distinct from whether it is statistically detectable at all; used in a synthesis matrix alongside a reported mean to judge whether a headline comparison between two methods reflects a meaningful practical gap or one that is technically present but small enough not to drive a real engineering decision on its own.
A standardized measure of how large a study's result actually was (commonly expressed in standard deviations), used to compare findings across studies and to avoid mistaking a modest result for a dramatic one.
The actual magnitude of a reported difference or improvement, as opposed to whether that difference cleared a statistical significance threshold. A statistically significant result can still have an effect size too small to matter for an engineering decision, per Section 3K.
Where it is taught
The exact lessons this term appears in. The first module of every program is free with a free account.
- What the Evidence Really Says About AI and Learning · Bonus: AI for Educators, Certified AI Practitioner: Workplace Foundations
- Reading a research paper and extracting what matters · Research, Benchmarking, and Staying at the Frontier, Engineering Judgment and Professional Formation
- Running a literature review on a hard problem · Research, Benchmarking, and Staying at the Frontier, Engineering Judgment and Professional Formation
Where it shows up at work
A definition is where Effect size starts. The lessons above teach it inside the competencies below, which are what job descriptions ask for and what the GAGE credential records. Every link here is drawn from the same careers map that builds a role's readiness check.
Designs tests for factuality, robustness, fairness, safety and abuse resistance with rubrics, baselines and thresholds, and says what a score misses.
One lesson on this page teach it as a primary outcome.
Explains training, tokens, context windows, embeddings, retrieval and fine-tuning well enough to ask an engineer a precise question and spot weak evidence.
One lesson on this page teach it as a primary outcome.
Reads bills and rules like a practitioner, compares options with costs and unintended effects, and writes briefs a decision-maker can act on.
One lesson on this page teach it as a primary outcome.
Roles that hire for it
Each role page shows the full competency set, what the seat decides, and a readiness check against your own evidence.
- Chief Technology Officer, AI engineering editionneeds it at depth
- AI Evaluation Specialistneeds it at depth · 6 open postings this week
- AI Model Validatorneeds it at depth
- Model Risk Managerneeds it at depth
- AI Product Manager, responsible product editionneeds it at depth · 3 open postings this week
- AI Security Architectneeds it at depth · 8 open postings this week
How job descriptions say it
Postings rarely use the textbook word. These are the phrases the careers map reads as asking for the competencies above.
Where the frameworks place it: NIST AI RMF MEASURE 1 and 2; NIST AI 600-1.
One interview question about it
Design the evaluation for a customer-service model before launch. What do you test, against what data, and what result blocks the release?
Asked of candidates for ai evaluation and testing design. Answering it in your own words is exactly how a GAGE topic is passed, and the credential records the pass where an employer can check it.
If you come from another field
Some backgrounds already carry this competency, and the readiness check gives credit for it.
- Model risk, credit risk and quantitative analysis carries it as work you can show: Test design and benchmark selection are the craft.
- A computer science or software engineering degree carries it as a claim to back up: Testing is second nature; AI evaluation design is a new shape of it.
- A data science, statistics or analytics degree carries it as a claim to back up: Evaluation design for a governance decision is a new frame for a known skill.
Terms it appears with
Not an alphabetical neighbourhood: these are the terms taught in the same lessons, ranked by how often they appear together.