Capability versus reliability

A capability is something a system can do in at least one case (what a demo shows); reliability is how often it does it correctly across inputs you did not choose (what an eval measures). Trust depends on reliability, and mistaking a demonstrated capability for reliability is the central error of the module.

The distinction between what an agent can attempt (almost anything phrased as a goal) and what it can reliably do on the real, messy version of a task (a separate, usually lower, empirically-measured number). Delegation depends on the reliability, not the attempt.

The core distinction of this topic. Capability is what an agent can do in its best moment (often shown in a demo); reliability is how consistently it does the task correctly, unsupervised, across varied real conditions. The gap between them is large in 2026.