GPAI upstream: what your foundation-model vendor owes you and what you must verify yourself
The short answer
You cannot govern the vendor's model, only the seam between you
You did not choose the training data and cannot audit it. What you can govern is the boundary: what the vendor legally owes you, what they delivered, and what no vendor document will ever cover, which is yours to verify.
What you will be able to do
- Distinguish the three roles the EU AI Act (Regulation (EU) 2024/1689) assigns along a model's life, the GPAI model provider, the downstream provider, and the deployer, and place your own organization in the right one for a specific system.
- List what a GPAI model provider legally owes you under Article 53 of the EU AI Act, and check each item against what your vendor actually delivered.
- Separate the obligations the vendor's documentation discharges from the obligations that remain yours no matter what the vendor sends, so you never mistake a model card for your own conformity work.
- Identify the upstream risks a vendor is not required to eliminate and often cannot, training-data provenance, copyright exposure, and privacy exposure, and decide what you must verify or contract for yourself.
- Analyze a vendor's capability and safety claims for reproducibility, refusing to inherit a number you cannot decompose into what was measured, on what data, judged by whom. (see Topic 4.2)
- Build a vendor documentation review that sorts owed-and-received from must-verify-yourself, and wire it into your conformity file and your evidence annex.
- Defend the boundary you drew to a regulator or a board that asks "why did you trust this vendor, and what did you not take on faith?"
The ideas, one by one
Your obligations flow from your role, so place it first
GPAI model provider, downstream provider, or deployer are roles you occupy per system, not fixed identities. The value-chain rules (Article 25) can turn a deployer into a provider through rebranding or substantial modification, so determine the role before anything else.
Article 53 owes you a finite, specific list
Downstream documentation on capabilities and limits, a copyright policy, a public training-content summary, and regulator-facing technical documentation. Knowing the exact list is what lets you check delivery instead of accepting whatever the vendor chose to send.
A model card is not your conformity work
The vendor documented a general model; you are answerable for a specific system it has never seen. Your intended-purpose testing, risk classification, oversight, and monitoring stay yours no matter how thorough the vendor's paperwork.
The training summary and copyright policy are a floor, not a clearance
The summary is broad by design and the policy is a commitment to a process. Neither certifies the data was lawful, which is exactly the question the New York Times case is fighting over with no resolution yet.
Provenance risk reaches your product three ways
Legal disruption that travels if the model is changed or withdrawn, regurgitation of protected content under your name, and outright unlawful content in the corpus. You inherit all three from a seat where you can see none of them.
Refuse to inherit a number you cannot decompose
A vendor's accuracy or safety claim becomes your liability the moment you rely on it. Decompose it into what was measured, on what data, judged by whom, or record it as the vendor's unverified assertion, never as your own fact. (see Topic 4.2)
Every open risk goes to test or contract, and none floats
Test what is about your system and checkable at your desk; contract for what turns on the vendor's conduct or unsettled law. An indemnity allocates who pays; it does not erase a risk you could have tested. An untagged risk is unmanaged.
The seam runs both ways
What your system sends upstream can be as damaging as what the model's training sends down. Know what leaves your walls, whether the vendor may train on it, and whether the data class is lawful to send at all. (see Topic 6.6)
The model you validated is not necessarily the model you run
Vendors change and deprecate models with little notice and behavior shifts between versions. Pin the version, monitor for change, contract for notice, and rerun critical evals on every version you are moved to. (see Topic 10.3)
The customer holds you, not the vendor
An airline answered for its chatbot's words because the customer dealt with the airline. Upstream duties on the vendor do not stand between you and the person your system harms, which is why the seam must be governed before anything goes wrong.
How the data was acquired can matter as much as whether training is fair use
A court found training on lawfully bought books transformative but using pirated copies not, ending in a record USD 1.5 billion settlement. So "how did you acquire your training data" is a sharper question to put to a vendor than "is your training lawful," and it is one the copyright policy does not answer.
You may not be buying from the model maker
Resellers, marketplaces, and aggregators wrap other companies' models, so your Article 53 documentation must trace to the real provider, and your version and provenance rows must point at the true source, not a wrapper that can change what it wraps without notice. (see Topic 3.3)
Read each Article 53 document for what it is
The Annex XII pack is a capabilities sheet, the training summary is a broad inventory, the copyright policy is a commitment to a process, and the regulator file is one you usually never see. None of them is your accuracy, your clearance, or your conformity work; each is a useful input from the vendor's side of the seam.
The vendor documentation review is a living artifact, not a form
Re-walk it on version change, on term change, and on material legal developments. It feeds your conformity file and your evidence annex, and it turns "we use a vendor's model" into a documented, defensible boundary. (see Topic 5.6) (see Topic 10.6)
Listen to it
The same lesson, talked through by two hosts as episode 38 of the podcast.
Real cases
These are real, documented cases from several jurisdictions, each showing the upstream relationship done well or badly. Each is owned for deep treatment by the topic noted and used here only for its upstream-governance angle.
Example 1: The New York Times v. OpenAI and Microsoft (United States, 2023 to present). The Times alleges the companies trained their models on millions of its articles without permission and that the models can regurgitate its content. Source: The New York Times, 27 December 2023. As of July 2026 the case is in discovery in the Southern District of New York, with the news plaintiffs seeking sanctions over OpenAI's handling of searches for the copied material. Source: reporting via The Washington Post, 9 July 2026. No final ruling has issued. The upstream lesson: a deployer built on a model at the center of an unresolved, high-stakes training-data dispute carries a provenance risk it did not create and cannot audit, which is precisely why the risk must be named, contracted around where possible, and monitored, not assumed away. This is the anchor case for the topic.
Example 2: Getty Images v. Stability AI (United Kingdom, decided November 2025). Getty sued the maker of an image-generation model over training on its photographs. On 4 November 2025 the High Court of England and Wales rejected the central copyright claim, accepting the view that the trained model contained no stored copies of the works, and found only limited, historic trademark infringement tied to old model versions that reproduced watermarks. Source: Getty Images v. Stability AI, High Court of England and Wales, November 2025; Bird & Bird and Latham & Watkins analyses, 2025. The upstream lesson: training-data disputes do not resolve the same way in every forum, and a deployer who assumes either "training is clearly infringing" or "training is clearly fine" is guessing at a question courts are answering differently by jurisdiction and by claim type. The governance response is to treat the legal question as genuinely open and allocate the risk deliberately.
Example 3: Thomson Reuters v. Ross Intelligence (United States, 2025). A federal court rejected a fair-use defense for training a legal-research AI on Westlaw headnotes. (see Topic 10.5) owns this case for the ownership-of-output angle; here it is the counterweight to Getty, showing that in a different forum, on different facts, a training-data use was found not to be fair use. The upstream lesson for a deployer: the outcome turns on specific facts and specific jurisdictions, so "another model won its case" is not clearance for the model you actually use.
Example 4: The withdrawn image-training dataset (international, 2023). Researchers found thousands of suspected illegal-content links inside a very large public dataset used to train popular image models, and the dataset was taken down. (see Topic 2.1) owns this case. The upstream lesson here: provenance risk is not only copyright. A model can be trained on material that is unlawful for reasons that have nothing to do with rightsholders, and a deployer inherits that too, which is why the training-summary and copyright-policy documents you receive are a floor, not a clearance.
Example 5: Samsung's confidential-code leak into a public chatbot (South Korea, 2023). Employees pasted proprietary source code into a public AI chatbot to debug it, and the company subsequently restricted use of such tools after concluding internal material had left the organization. Source: reporting via Bloomberg and TechCrunch, 2023. The upstream lesson: the seam runs in both directions. What your workflow sends up to a vendor can be as damaging as what the vendor's training data sends down, and governing only the download while ignoring the upload covers half the boundary.
Example 6: The airline chatbot held to its word (Canada, 2024). A tribunal found an airline liable for wrong information its website chatbot gave a grieving customer, rejecting the claim that the chatbot was a separate responsible entity. Source: Moffatt v. Air Canada, British Columbia Civil Resolution Tribunal, 14 February 2024. The upstream lesson: the customer holds the deployer responsible for the output, no matter whose model produced it. Upstream duties on the vendor do not stand between you and the person your system harms.
Example 7: The EU General-Purpose AI Code of Practice (European Union, 2025). On 10 July 2025 the European Commission published a voluntary Code of Practice with three chapters, Transparency and Copyright for all GPAI providers, and Safety and Security for systemic-risk providers only, as a way to demonstrate Article 53 and 55 compliance until harmonised standards exist. Source: European Commission, digital-strategy.ec.europa.eu, 10 July 2025. The upstream lesson: whether your vendor has signed the Code, and which chapters, is a concrete piece of due-diligence intelligence about how the vendor is meeting its obligations, and a reasonable question to put to them in writing.
Example 8: A regulator forcing a training pause (Brazil, 2024). A data-protection authority ordered a large platform to stop training AI on its users' personal data, with a daily penalty for non-compliance. (see Topic 6.6) owns this case for the cross-border decision. The upstream lesson: the lawfulness of feeding personal data into a model is contested and enforced differently across jurisdictions, so a deployer sending personal data upstream, or relying on a model trained on it, faces a moving legal floor and cannot treat the vendor's home-country practice as a global permission.
Example 9: Bartz v. Anthropic, the pirated-books settlement (United States, 2024 to 2026). Three authors sued the maker of the Claude models, alleging it trained on their books copied from pirate "shadow libraries." In June 2025 Judge William Alsup of the United States District Court for the Northern District of California drew a sharp line: training on lawfully acquired books was, in his words, "quintessentially transformative" and protected as fair use, but downloading and keeping pirated copies of those books was not. Source: Bartz v. Anthropic, N.D. Cal., summary judgment June 2025; Kluwer Copyright Blog and Authors Guild analyses, 2025. In September 2025 the company agreed to pay a minimum of USD 1.5 billion, on the order of USD 3,000 per work, and to destroy the pirated copies; a final fairness hearing was held in May 2026, the court granted final approval in July 2026, and initial payments were estimated to begin around August 2026. Source: Authors Guild, "What Authors Need to Know About the Anthropic Settlement," 2025; Publishing Perspectives and court reporting, 2026.
The upstream lesson, and the reason this belongs beside the New York Times and Getty cases, is that the provenance risk did not turn on the abstract question "is training fair use." It turned on how the vendor obtained the data: the same court that blessed training on bought books condemned training on pirated ones. A deployer cannot see acquisition method from its seat, which is exactly why "how did you acquire your training data" is a sharper written question to put to a vendor than "is your training lawful," and why the copyright-policy document you receive, a commitment to a process, does not answer it. It also shows the scale of an upstream provenance failure: a record settlement whose disruption, had this been a smaller model many products stood on, would have rippled into every system built above it.
Example 10: The whistleblower who questioned his own employer's data (United States, 2024). A former researcher at a major AI lab publicly argued the company's training practices infringed copyright and offered documents. (see Topic 11.5) owns this case, and it must be handled with care. The upstream lesson referenced here only: even insiders dispute the lawfulness of the training data behind widely deployed models, which tells a deployer that the provenance question is genuinely unsettled at the source, and confident public assurances from a vendor are not the same as a resolved legal position.
Where people go wrong
Most of these are errors of role and of assumption, not of intelligence. A capable team makes them because the upstream relationship feels like someone else's problem until it lands on the deployer's desk. Read each as a way your own next vendor relationship could quietly leave a risk unassigned.
- "We do not train models, so the training-data lawsuits do not touch us." They touch you through your dependency. If the model at the center of a dispute is changed, restricted, or withdrawn, your product built on it inherits the disruption, and if it regurgitates protected content, the emission happens in your product under your name. Not training the model removes you from the defendant's chair, not from the risk.
- "The vendor's model card and our contract cover our obligations." They cover the vendor's account of the model, not your responsibility for your system. The vendor documented a general model; you are answerable for a specific use it has never seen. Your intended-purpose testing, risk classification, oversight design, and monitoring stay yours no matter how thick the vendor's documentation is.
- "We are a deployer, so our duties are light." Only if you are actually a deployer for this system. If you put your name on the product or substantially modify the model, the Act's value-chain rules (Article 25) can make you a provider with full obligations. Assuming the lighter role without checking is how deployers leave provider-level duties unassigned.
- "The training summary tells us the data was clean." The Article 53 training summary is a broad, templated overview, explicitly not a full manifest, and it certifies nothing about lawfulness. The copyright policy it comes with is a commitment to a process, not a clearance of the corpus. Reading either as proof the data is safe overreads a document the law designed to be summary-level.
- "The vendor passed adversarial safety testing, so our system is safe." The systemic-risk provider tested the model in general against certain attacks. It did not test your system, your prompts, your users, or your trust boundary. A model can be broadly robust and still fail your specific high-stakes case, and the provider's evaluation runs to the regulator, not into your conformity file as your evidence. (see Topic 4.4)
- "We can cite the vendor's accuracy and safety numbers as our own." A number you cannot decompose into what was measured, on what data, judged by whom, is a liability the moment you rely on it. Vendor claims are the vendor's assertions until you verify them, ideally by rerunning the relevant check on your representative data. Inheriting a number undecomposed imports an exposure you will have to defend and cannot. (see Topic 4.2)
- "Provenance risk is a copyright problem." Copyright is one strand. Training data can carry privacy exposure, personal data used without a lawful basis, and outright unlawful content, as the withdrawn image dataset showed. (see Topic 2.1) Scoping the upstream risk to copyright alone misses the privacy and illegal-content strands a deployer also inherits.
- "Whatever we send the model stays private." What the vendor does with your inputs is set by their terms and your configuration, not your intentions. A default consumer tier may log, retain, or train on what you send, and a real company restricted a public chatbot after employees leaked confidential code into it. The upload half of the seam is a governance surface, not an afterthought.
- "The model we validated is the model we run." Vendors update, retrain, and deprecate models, sometimes with little notice, and behavior can shift sharply between versions. (see Topic 10.3) Conformity evidence gathered on version A describes a system you no longer run once the vendor moves you to version B. Version is a first-class fact to pin, monitor, and contract notice for.
- "An indemnity in the contract means the risk is gone." A contract does not make a risk disappear; it decides who pays when it lands. An indemnity is valuable and it is not a substitute for testing the risks you can close yourself, nor a guarantee the counterparty can actually make you whole. Contract and test are two different paths, and each open risk needs one of them, explicitly.
- "The open-source model has no vendor obligations, so we are on our own with no recourse but also no upstream duties." Open-source GPAI models are exempt from some Article 53 documentation duties, but not from the copyright policy or training summary, and not at all if they carry systemic risk. And your own downstream obligations do not lighten because the model is open. Open weights change the documentation you receive; they do not change that the deployed system's governance is yours.
- "Because another model won its copyright case, ours is cleared." Training-data disputes resolve differently by jurisdiction and by claim type; one forum accepted that a model holds no stored copies while another rejected a fair-use defense on different facts. A favorable ruling for a different model in a different court is not clearance for the model you actually deploy. Treat the legal question as genuinely open and allocate the risk deliberately.
- "Sorting the seam once is enough." The boundary moves when the model beneath you moves, when the vendor changes terms, and when the law develops. A vendor documentation review is a living artifact re-walked on version change and on material legal developments, not a one-time form. A boundary drawn once and never revisited describes a system and a legal landscape that have both moved on.
- "Our vendor is the model maker, so our documentation comes from them." Often you are not calling the model maker at all but a reseller, cloud marketplace, or aggregator that wraps someone else's model. That middle party cannot hand you Annex XII documentation it never wrote. Your Article 53 documents must trace to the actual model provider, and a reseller who cannot point you to them is a documentation gap, not a discharge. Ask who is really behind the interface. (see Topic 3.3)
- "Fair use was found for training, so the provenance question is settled." One court found training on lawfully acquired books transformative while holding that using pirated copies was not, and other forums have ruled differently again on different facts. The lawfulness can turn on how the data was obtained, not just on the abstract fair-use question, and a deployer cannot see acquisition method from its seat. Treat the question as genuinely open and ask the vendor how, not just whether, it acquired its training data.
- "Light fine-tuning keeps us a deployer." Maybe, maybe not. The more you change the model's intended purpose or behavior, the further the value-chain rules (Article 25) push you toward provider duties, and there is no bright line for "substantial." When the case is genuinely ambiguous, carry the heavier role until counsel confirms the lighter one, because the dangerous error is to assume down and leave provider-level duties unassigned.
- "The Annex XII pack tells us our accuracy is fine." The downstream documentation reads like a capabilities sheet for a general model: what it is good at, its known limits, its intended uses. It cannot contain your accuracy on your task, because it has never seen your task. Reading a general capabilities description as evidence about your specific decision is the exact seam error, moving a duty from your side to the vendor's by wishful reading.
Questions people ask
- What is general-purpose AI model (GPAI model)?
- A model trained on broad data that can perform many tasks and be integrated into many different downstream systems, for example a large language model. The EU AI Act gives GPAI model providers a specific set of obligations under Articles 51 to 56. More on General-purpose AI model (GPAI model)
- What is foundation model?
- A common industry term for a large, general-purpose model trained on broad data and adapted to many uses. Overlaps in practice with the EU AI Act's "GPAI model," which is the term the law actually uses. More on Foundation model
- What is GPAI model provider?
- The organization that develops a general-purpose AI model and places it on the market. It carries the Article 53 documentation, copyright, and training-summary obligations, and, for the largest models, the Article 55 systemic-risk obligations. Usually not the deployer.
- What is downstream provider (provider of an AI system)?
- An organization that builds an AI system, often on top of a GPAI model, and places it on the market under its own name. It carries the full obligations attaching to the system's risk class, and can become a provider through the value-chain rules even without training a model.
- What is deployer?
- An organization that uses an AI system under its own authority in the course of its work, without placing it on the market. Deployers carry duties such as human oversight, use per the instructions, and, for high-risk systems, logging and cooperation. More on Deployer
Keep going
This lesson builds Contract terms that allocate AI risk, and that page shows the roles that hire for it. Every Certified AI Governance Professional (CAIGP) lesson.
Watch the lesson and prove it
Proving it happens inside the program: watch, explain it in your own words, and it goes on a record employers can check.
Watch the lesson and prove it →