Reading the primary source: a model card, a system card, and what they do not say
The short answer
The card is a primary source written by an interested party
Read it the way the International AI Safety Report (2025) read the science, marking confidence, except knowing the confidence markers were chosen by the seller. That makes reading them correctly more important, not less.
What you will be able to do
- Read a real vendor model card or system card end to end and locate its load-bearing sections (intended use, out-of-scope use, evaluations, limitations, training-data description, and change or version history) without being led by its framing.
- Separate three registers inside any such document: a firm claim backed by stated evidence, a hedge that sounds like a claim but commits to nothing, and a silence where an operator-relevant fact should be and is not.
- Analyze the "intended use" and "out-of-scope use" language for how it shifts the burden of safe operation onto you, the deployer, and decide what that shift actually obligates you to test.
- Apply the evidence-dilemma stance from the International AI Safety Report (2025): treat "the evidence does not establish this" as information you can act on, not as a gap to paper over.
- Detect staleness: find the date and version, decide whether the card still describes the system you are calling today, and connect this to documented behavior drift (see Topic 10.3).
- Cross-check a card's claims against independent sources rather than trusting the document alone, and know which claims you must reproduce yourself (see Topic 12.2).
- Produce a one-page source-reading note on the vendor card behind your own highest-stakes AI system, listing verified claims, unverifiable claims, and the specific questions to send the vendor (see Topic 3.3).
- Distinguish what a model card can tell you from what only a system card or your own testing can, so you never let a document about the model stand in for evidence about the deployed system.
The ideas, one by one
Every line is a claim, a hedge, or a silence
A claim names something checkable. A hedge sounds firm and commits to nothing. A silence is the operator-relevant fact that is not on the page. Sorting a card into these three registers is the whole skill; the silence list is usually the most valuable output.
Silence is not safety
A card that says nothing about your language, your use, or your failure mode has told you nothing, which for a decision is the same as a no. Absence of a claim is never evidence of safety.
Out-of-scope language shifts liability to you
The uses a vendor refuses to stand behind are on record as your risk if you deploy them anyway. Read that section to learn whether your real use is inside the bounds someone actually validated.
Check the version and date first
Vendors update models behind stable names with no changelog, and behavior drifts (see Topic 10.3). A card older than the last update may describe a model you no longer call. That uncertainty belongs at the top of your silence list.
A close reading does not resolve uncertainty, it makes it precise
The output is not a yes or no; it is a sorted list of claims to reproduce yourself (see Topic 12.2) and questions to put to the vendor in writing (see Topic 3.3). A model card can never substitute for testing the deployed system.
The reading is an artifact, and it persists
Your one-page source-reading note goes into the evidence annex (see Topic 10.6), makes your standing frontier watch cheap (see Topic 12.3), and travels to your successor so your understanding outlives your tenure (see Topic 12.5).
A specific limitations section is a trust signal, a generic one is a red flag
"The model fabricates policy details for niche queries" hands you something to test; "outputs should be reviewed" protects the vendor and helps you not at all. Grade a card by how much of its bad news is specific enough to act on.
A close reading does not replace testing, it aims it
The whole payoff is a short, precise list of the few load-bearing claims to reproduce on your own data and the exact facts to ask the vendor, so you spend effort on what carries the decision instead of on everything or nothing.
Listen to it
The same lesson, talked through by two hosts as episode 90 of the podcast.
Real cases
These are real documents and real events, used to show the reading discipline in action. Read each for the claim, the hedge, and the silence.
Example 1: The International AI Safety Report (2025), the standard to read toward. The first full report, led by Yoshua Bengio and backed by the 30 nations of the Bletchley process, is the clearest public example of a primary source that marks its own confidence honestly. It separated risks with robust empirical evidence (for example, harms from AI-generated media) from risks resting on modeling and theory (for example, some future-capability scenarios), and it stated plainly where substantial uncertainty remained. It made no policy recommendations, an act of discipline, refusing to claim more authority than a synthesis of evidence can carry. (UK Government, "First International AI Safety Report," 2025; a second edition followed in 2026, and the confidence-marking discipline it models carried over.) When you read a vendor card, you are reading toward this standard and measuring the distance the card falls short of it.
Example 2: The Google model card proposal (Mitchell and colleagues, 2019). The original "Model Cards for Model Reporting" paper introduced the structure operators now read: model details, intended use and out-of-scope use, factors, metrics, evaluation and training data, disaggregated quantitative analysis, and caveats (Mitchell et al., ACM Conference on Fairness, Accountability, and Transparency, 2019). Its motivating example was a face-detection model whose aggregate accuracy hid large performance gaps across gender and skin tone. The lesson for a reader: the disaggregated breakdown is the load-bearing section, and its absence in a real card is a silence, not a simplification.
Example 3: The OpenAI GPT-4 System Card (2023). When OpenAI published the GPT-4 System Card in March 2023, it documented not just capabilities but red-team findings and residual risks after mitigation across areas including bias, disinformation, over-reliance, privacy, cybersecurity, and proliferation. It is a genuinely substantive system card, and reading it teaches both halves of the skill: the claims are specific and useful, and the silences (for example, limited detail on training-data composition) are real and worth listing. A strong card still has silences; the reader's job does not end because the document is good.
Example 4: Anthropic frontier system cards (2025 onward). By late 2025 and into 2026, frontier developers were publishing lengthy system cards running to hundreds of pages, for example the Claude Opus 4.5 system card (Anthropic, November 2025), documenting safety evaluations, misalignment testing, and residual behaviors in unusual detail. The reading lesson flips: when a card is very long and very detailed, the risk is not a thin document but reader fatigue, that the one operator-relevant limitation is buried in a hundred pages of evaluation. Close reading scales with the document; a long card demands a checklist, not more patience.
Example 5: A claimed metric is not a reproduced metric. The gap between a metric a vendor states in a card and the same metric measured independently is not academic; it is the gap regulators now litigate and the gap external validation keeps exposing. This program treats two such cases in depth: a state enforcement action over a healthcare vendor's misrepresented accuracy claims (see Topic 4.2), and an external validation of a widely deployed clinical model whose real-world performance was far worse than its reputation (see Topic 4.6). For a reader here, the shared lesson is narrow and durable: read every metric in a card as a claim to be reproduced on your own use, not a fact to be believed.
Example 6: The EU AI Act GPAI documentation duty (in force 2 August 2025). The Act requires GPAI model providers to give downstream providers the information they need to understand a model's capabilities and limitations and to publish a summary of training content (Article 53, Regulation (EU) 2024/1689). This is the legal system formalizing exactly the reading you are learning: the law now presumes a downstream operator will receive and use vendor documentation to meet their own obligations. The card is no longer optional to read; a whole compliance posture now assumes you read it.
Example 7: Policy documents read the same way. The skill transfers past vendor cards to every document an operator meets. A soft-law declaration on frontier-AI safety, rich in shared intention and deliberately thin on binding commitment, is treated as its own topic in this program (see Topic 6.5); reading one teaches the same discrimination you apply to a model card, separating what was agreed and enforceable from what was merely affirmed. A regulator's guidance, a standards draft, a benchmark paper: each yields to the claim, hedge, and silence sort, so the habit you build on a model card pays off across your whole reading life.
Example 8: The GPT-4o System Card (2024), a card that grew with the capability. When OpenAI published the GPT-4o System Card in August 2024, the model was multimodal, including a voice capability, and the card added risk categories the earlier text-only cards did not need, covering areas such as voice-related risks and the potential for users to form emotional reliance on a human-sounding system. The reading lesson is that a card's structure tracks the capability: as a model gains a new modality, new sections appear, and the absence of a section that a new capability clearly demands is itself a silence. When you read the card for a capability the vendor just shipped, check that the card actually grew to cover it rather than reusing last version's template.
Example 9: The model-card ecosystem on public model repositories. Model cards have become a standard artifact on public model-hosting platforms, where thousands of models each ship with a card template. This is the good news (documentation is now expected) and the trap (a present card is not a complete card). Many community cards are auto-generated stubs with the honest-bad-news sections left blank, so the existence of a file named "model card" tells you nothing about whether it answers an operator's questions. The reading discipline is identical whether the card is a two-hundred-page frontier document or a three-line stub: apply the same checklist and let the silences fall out. A blank limitations section on a stub is exactly the same finding as a missing one on a polished card.
Example 10: When there is no truthful card at all, the AI-washing pattern. The sharpest lesson about reading a primary source comes from the cases where the primary source was a lie or did not exist. In a run of enforcement actions, regulators found companies whose public claims about an "AI" product did not match a system that was, in reality, largely run by people, and the paper trail of overstated capability was central to the case (the AI-washing pattern treated in Module 4 and the vendor-interrogation case in Module 3). (see Topic 4.1) (see Topic 3.3) For a reader, the takeaway is that the absence of a truthful, specific card is not a neutral gap; it is the same condition that precedes the enforcement actions, because a company that will not document what its system actually does is often a company whose marketing has outrun its engineering. A missing card is a maximal silence, and a silence that regulators have repeatedly found sitting on top of a misrepresentation. The retail sector produced its own version of the pattern in the widely reported case of an automated checkout system marketed as computer vision that turned out to depend heavily on remote human review (see Topic 0.2); no card or claim survives contact with the actual headcount behind the "automation," which is exactly why the reading discipline in this topic treats a vendor's own description as a claim to verify, never a fact to accept.
Where people go wrong
- "The card is the vendor's official statement, so I can rely on it." A card is a document written by the party that most wants you to say yes. Relying on it is not the same as reading it. The International AI Safety Report was written by independent experts mandated to mark uncertainty; a vendor card is written by the seller. Read the vendor card knowing who wrote it and why, and treat every metric as a claim to verify, not a fact to accept.
- "A confident, polished card is a trustworthy card." Polish measures the writing budget, not the honesty. A thin or generic limitations section ("as with all AI systems, review outputs") is a warning sign; a specific, uncomfortable limitations section ("fabricates policy details for niche queries") is a sign of a card you can actually use. Trustworthiness lives in specificity and in the willingness to state bad news, not in fluency.
- "No news is good news, if the card does not mention a risk, there is not one." Silence is the most dangerous register, not the safest. A card that says nothing about non-English performance has not told you it is safe in other languages; it has told you nothing, which for your decision is the same as a no. Absence of a claim is never evidence of safety. Keep a silence list precisely because silences do not announce themselves.
- "A hedge is basically a claim." "Designed to be safe," "strong performance," and "significant mitigations" are engineered to sound like claims while committing to nothing a test could falsify. Reading a hedge as a claim is how organizations deploy on documents that never actually said what they needed. When you meet a hedge, rewrite it in the margin as the question it is dodging.
- "A model card is enough to approve a deployment." A model card describes the model in the lab. Your risk lives in the deployed system: the prompt, the filters, the human steps, your data, your users. A model card's silence on system-level behavior is not covered by reading it harder; it is closed only by a system card (see Topic 10.3) or by your own testing. Never let a document about the model stand in for evidence about the product.
- "If the card cites a benchmark score, the capability is established." A benchmark can be unrepresentative of your use, contaminated by test data leaking into training, or funded by an interested party, all live problems at the frontier (see Topic 12.2). A cited score is a claim about a test, not proof of fitness for your purpose. Reproduce the claims that matter on your own data.
- "The version does not matter, it is the same product I evaluated last quarter." Vendors update models behind stable product names, and behavior can shift sharply with no changelog, a documented phenomenon (see Topic 10.3). A card without a build version, or a card older than the last update, may describe a model you no longer call. Check the date and version first, not last.
- "Out-of-scope use is legal boilerplate I can ignore." The out-of-scope section is a quiet transfer of liability: it puts on record the uses the vendor refuses to stand behind, which means if your use is out of scope and it fails, the vendor already warned you. Ignoring it does not make your use covered; it makes you the party holding the risk. Read it to find out whether your actual use is inside the bounds someone validated.
- "An AI tool summarized the card for me, so I have read it." A tool's summary of a card is a secondary source built on top of a secondary source, and it will smooth over exactly the hedges and silences you are trying to catch, because summarizers reward fluent generalities. Use a tool to speed the first sort if you like, but verify its output against the actual document, and never let the summary become the thing you file. The prompts in this topic are aids to your reading, not replacements for it.
- "The card cites a certification, ISO/IEC 42001 or a NIST AI RMF profile, so it has been independently checked." A management-system certification like ISO/IEC 42001 (established 2023) certifies that an organization runs a governance process, not that a specific model behaves as the card claims; it is a voluntary, certifiable standard, not a mandate, and not a test of your use case. A NIST AI RMF profile is a structured self-assessment against a framework, not an outside party grading the card's claims. Read a cited certification as evidence the vendor has a process, and keep reproducing the specific claims that matter to your deployment regardless.
- "The card is honest, so I can deploy without my own testing." Honesty is not sufficiency. Even a scrupulously honest card describes the vendor's tests, not your data, your users, or your edge cases, and it cannot know your deployment context. A close reading of an honest card still ends in a list of claims to reproduce; the honesty just means the list is shorter and the reproduction more likely to confirm. Reading grades the document; testing grades the fit.
- "If I ask the vendor a question and they do not answer, I have learned nothing." A declined or evasive answer to a specific, reasonable written question is itself evidence, and you record it as such. A vendor who will not put the training cutoff or the evaluated languages in writing has told you the fact is either unknown to them or uncomfortable to state, and either way that shapes how much weight your deployment can put on the model. Silence from a vendor is read the same way as silence in a card.
Questions people ask
- What is model card?
- A short, structured document that travels with a trained model and states what it is, what it is for, where it works, where it fails, and how the vendor knows. Introduced formally by Mitchell and colleagues (2019). For a reader, the primary source that describes the model in the lab, distinct from the deployed system. More on Model card
- What is system card?
- A document describing a deployed system built around one or more models, including guardrails, filters, human-oversight steps, and residual risks after mitigation. Where a model card describes the model, a system card describes the product a user actually touches. (see Topic 10.3) More on System card
- What is primary source?
- The original document a claim comes from, as opposed to a summary, ranking, or commentary built on top of it. In applied AI the primary source is usually the vendor's model card or system card, or a foundational report; frontier discipline means going to it directly. More on Primary source
- What is claim?
- A firm assertion backed by stated evidence that could in principle be checked or reproduced, for example a named benchmark score under a named condition. The parts of a card an operator can actually use, once traced to evidence. More on Claim
- What is masked claim?
- A sentence that names a number but strips out the condition (the test, the split, the population) that would make the number checkable, for example "91.2 percent accuracy" with no benchmark named. Not a hedge, because it commits to a figure, but unusable until the missing condition is found or requested; treat it as a hedge until then.
Keep going
This lesson builds Content provenance and authenticity, and that page shows the roles that hire for it. Every Certified AI Governance Professional (CAIGP) lesson.
Watch the lesson and prove it
Proving it happens inside the program: watch, explain it in your own words, and it goes on a record employers can check.
Watch the lesson and prove it →