Benchmark

A standardized test or task used to measure and compare AI model performance on a specific, defined capability, such as a set of exam questions or coding problems. A strong benchmark result demonstrates performance on that specific, bounded task; it does not, by itself, establish general real-world capability, as Section 3J's worked example demonstrates.

A shared, public test set used to compare models against each other in general. Useful for ranking, almost never containing your domain, users, or incident, and sometimes compromised by who funded it or who saw the questions, so it is not a substitute for your own suite.

A standardized test used to score and compare systems. It is evidence only once you can answer whose test it is, who chose or funded it, and whether the scored system is the one you would deploy; otherwise it is a number that functions like a demo.

A fixed set of tasks with known answers used to measure and compare AI models. A benchmark score is the outcome of running a model against those tasks under a specific procedure, not an intrinsic property of the model.