Skip to main content

Third-party data and the vendor claims you must verify yourself

The short answer

You inherit the liability, not the excuse

Buying or licensing third-party data transfers the data, its utility, and its risk to you, but not the accountability for the outcome. Under the GDPR's controller and processor structure (Article 28), the duty to verify a vendor's guarantees is legally yours, and "the vendor told us" is not a defense the law offers.

What you will be able to do

  • Explain why buying or licensing third-party data transfers the data to you but does not transfer the accountability away from you, using the controller and processor structure of data-protection law.
  • Decompose a vendor claim (for example "fully consented," "de-identified," "ethically sourced") into the specific evidence that would prove or disprove it, which is the core Analyze skill of this topic.
  • Distinguish a verifiable claim (one you can check against a document, a sample, or a test) from an unverifiable assertion (one that has no evidence you could ever inspect).
  • Trace a third-party dataset back through its supply chain to its point of collection and its labor chain, and name the links where governance most often breaks.
  • Select the right verification method for each claim: document review, data sampling, an independent test, a reference check, or a site or process inspection.
  • Price the residual risk of a claim you cannot fully verify, and decide when to add a contract clause, when to escrow evidence, and when to walk away.
  • Trace a purchased dataset past the vendor in front of you to its enrichment link, detecting silent joins and laundered inferences that a single-source sales sheet conceals.
  • Produce a third-party data verification record that an auditor could read and that feeds your data provenance file.

The ideas, one by one

The sales sheet is a set of claims, not a set of facts

Every marketing adjective ("consented," "de-identified," "ethical," "compliant," "secure," "representative") is a compressed body of evidence you have not yet seen. The governor's job is to treat each one as a hypothesis and design the test, not to decide whether to believe the vendor.

Analyze means decompose the claim into evidence

For each claim, write the precise assertion, the evidence that would prove it, and the evidence you have actually seen. The gap between the last two is the finding. A claim whose proving evidence you have not obtained is not verified, however sincere the vendor sounds.

Trace the chain to the collection and labor links

Third-party data is many hops from its origin, and governance breaks most at the point of original collection (where consent and lawful basis live) and the labor link (where the Roomba images leaked). A vendor that will not name its chain is telling you where the risk is.

Match the verification method to the claim and the consequence

Document review, data sampling, independent test, reference check, and site inspection each prove different claims at different depths. Using a weak method on a high-consequence claim is how audits get faked. Proportionality is the judgment.

Price the residual, do not pretend it away

When you cannot fully verify a claim, choose accept-and-document, contract-the-gap, escrow-the-evidence, or walk-away, with a reason and a named owner. This turns "we did not know" into "we knew, we priced it, and here is who decided," which is the sentence that survives an audit.

The verification record is evidence, not paperwork

It consumes your AI systems inventory and data audit, and it feeds your data provenance file. When your data is challenged by a regulator, a plaintiff, or a hostile board, this record is the page that holds or folds.

A clean vendor is the goal, not a defeated one

Verification exists to tell a vendor whose adjectives are backed by evidence from one whose adjectives are backed by hope, and to prove later that you could tell. Reflexive rejection wastes good data as surely as reflexive trust ships bad data.

The quiet link is enrichment, not just collection and labor

Between where data is collected and where it is labeled, it is often joined to other sources and stamped with inferred attributes. A join inherits every parent's consent problems, and an inferred sensitive attribute is still sensitive. Ask whether the product is a single source or a merge, and what was inferred from what.

Verification is dated, not one-time

A claim that was unverified at purchase can surface as enforcement years later, as the multi-year location-broker cases show. The residual-risk decision, recorded with a date and an owner, is what protects you across that gap; "anonymous per vendor" ages into "we had nothing" the day the letter arrives.

The frameworks are on your side

NIST AI RMF, ISO/IEC 42001, and the AIGP body of knowledge all name third-party and supply-chain data as a distinct risk to manage, so verifying a data vendor is not obstruction; it is the recognized practice these frameworks expect, and the verification record is the artifact that proves you did it.

Listen to it

The same lesson, talked through by two hosts as episode 15 of the podcast.

Real cases

These are real cases of third-party data claims that did not survive contact with reality, sourced across regions. Each is analyzed for the claim that failed and the verification that would have caught it.

Example 1: iRobot Roomba development images and the labeling vendor (2022, global). This is the anchor case for this topic. iRobot, maker of the Roomba robot vacuum, ran a development program in which special pre-production robots, marked with bright green "video recording in progress" stickers and given to paid collectors and employees who signed agreements, captured images inside homes to improve the product's computer vision (MIT Technology Review, 2022; iRobot statements). Images from those homes, in the United States, Japan, France, Germany, and Spain, were captured between roughly June and November 2020 and sent to a data-labeling vendor, which used gig workers to annotate them. At least 15 of those images, including one of a young woman on a toilet and one showing a minor, were posted by workers to closed social media groups. After MIT Technology Review contacted iRobot, the company said the leak violated its agreements with the service provider and terminated the relationship; the labeling vendor said 13 of the 15 images came from an iRobot research project. The claim that failed was not "high quality data," it was "securely handled by our vendor and its workers." The verification that would have caught it was a process inspection of the labor link and a hard control on how far down the chain raw images were allowed to travel, plus knowing and authorizing every sub-processor rather than trusting a single vendor's assurance. The teaching point is stark: the data was collected with signatures and stickers, and it still became a privacy catastrophe, because the accountability could not be subcontracted along with the labeling.

Example 2: FTC actions against location-data brokers (2024, United States). In December 2024 the FTC took action against the data brokers Gravy Analytics and its subsidiary Venntel, and separately against Mobilewalla, over the sale of sensitive precise-location data (FTC press releases, 2024). The FTC alleged the brokers tracked consumers to sensitive places (health clinics, places of worship, political gatherings) and sold that data, and that Mobilewalla had collected more than 500 million unique advertising identifiers paired with precise location between January 2018 and June 2020, building audience segments including, in one reported instance, pregnant women identified from visits to pregnancy centers. The proposed orders barred the sale of sensitive location data and required deletion of historic data and any products derived from it. For a buyer, the failed claim was "de-identified" or "anonymous" location data; precise location pings are re-identifiable to an individual home and routine, which is the sampling-and-test finding a buyer could have produced in an afternoon. Any organization that had licensed this data as "anonymous" inherited the exposure the moment the enforcement landed.

Example 3: The labor link as a supply-chain risk (global). The Roomba leak sat at the data-labeling labor link, and that link is a recurring third-party risk far beyond one company. Data labeling is frequently subcontracted through multiple layers, and the more layers a chain has, the harder any single layer is to name, control, and audit, regardless of where those layers sit. The risk driver is the number of unaudited hops and the absence of a named, controllable sub-processor at each one, not the location of the workforce; a well-run, audited labeling operation is safe wherever it sits, and an unaudited one is a liability wherever it sits. The governance lesson is not that any particular labor market is risky; it is that the labor link is a real sub-processor with real access to your data, and "our vendor handles labeling" is a claim that hides a chain you are still accountable for. Verification means naming the chain, controlling what data reaches it, and inspecting how it is handled, not assuming that a contract with the top of the chain reaches the bottom. A practical control that would have blunted the Roomba failure is data minimization at the labor link: if raw images that may show people are never sent down the chain in the first place, redacted, masked, or replaced with a synthetic stand-in before labeling, a leak at that link exposes far less. Verifying the labor link and limiting what reaches it are two halves of the same governance move.

Example 4: The re-identification test as routine verification (established method). Across jurisdictions, regulators and researchers have repeatedly shown that datasets sold as "anonymized" can be re-identified by combining a few attributes, which is why data-protection authorities treat "de-identified" and "anonymized" as different legal standards. The durable, transferable point for a governor is that "de-identified" is a testable claim: sample the data, attempt re-identification with reasonable auxiliary information, and measure. A vendor's word that data "cannot be traced back" is a hypothesis you can falsify at your own desk, and the FTC broker cases show the consequence of not bothering. The reason this test is worth building into routine practice is its asymmetry: it is cheap to run (an afternoon on a sample) and the failure it catches is expensive to inherit (a re-identifiable dataset that a regulator later reclassifies as personal data you had no basis to hold). Few verification steps have a better ratio of effort to risk retired.

Example 5: The Kochava location-data case, verification as a slow-burning liability (2022 to 2026, United States). In 2022 the FTC sued the data broker Kochava over the sale of consumer location data that could be traced to sensitive places, including reproductive health clinics, and the matter ran for years before a settlement was reported in 2026 (FTC v. Kochava, FTC legal library). The teaching value for a buyer is the timeline: a "de-identified location data" claim that looked fine at purchase became a multi-year public liability, and any downstream organization that had licensed the feed as "anonymous" was exposed for the whole duration. Verification is not only a gate at purchase; a claim that was unverified at ingestion can surface as enforcement years later, which is why the residual-risk decision, dated and owned, matters as much as the initial check. A buyer who had recorded "location data, de-identification unverified, accepted because assumed low-risk" would at least have a dated decision to point to; a buyer who wrote "anonymous per vendor" has only the vendor's adjective when the letter arrives.

Example 6: The disclosed-corpus contrast, third-party model data you can partly verify (established, global). Not all third-party training data is a black box. Since 2 August 2025, providers of general-purpose AI models in the European Union have been obliged to publish a summary of the content used to train them (EU AI Act, established; upstream treatment owned by (see Topic 5.5)), and some model providers publish model cards describing data sources and known limitations. For a deployer adopting a pretrained model, these disclosures are the document-review evidence available where sampling the corpus is impossible: they do not prove the training data is clean, but they let you record what the provider claims, check it against the litigation record, and price the undisclosed remainder as an explicit residual risk. The contrast with a vendor that says only "trained on a large, high-quality corpus" is instructive: one gives you something to verify and something to watch, the other gives you an adjective. Prefer the third-party data you can partly verify over the third-party data you cannot see at all, and record the difference.

Example 7: The positive contrast, a vendor that survives verification. Not every third-party data relationship is a trap, and the point of verification is to find the vendors worth trusting, not to refuse all of them. A vendor that can hand you the actual consent-form wording, name and let you audit its labeling subcontractors, produce a distribution report on the dataset, warrant its claims in the contract, and pass a re-identification test on a sample is a vendor whose sales sheet turned out to be true. That is the outcome verification is for. The verification record does not exist to catch liars; it exists to let you tell the difference between a vendor whose adjectives are backed by evidence and one whose adjectives are backed by hope, and to prove, later, that you could tell.

Read across these cases, one pattern holds. In none of them did the failed claim announce itself; each looked fine on the sheet and became a liability only downstream, sometimes years later. The Roomba images were collected with signatures and stickers. The location data was sold as "anonymous." The enriched segments were sold as "behavioral." In every case the evidence that would have exposed the gap was obtainable before purchase (a process inspection, a re-identification test, a question about the join), and in every case the organization that skipped it inherited the consequence. That is the whole argument for the verification record: not that vendors lie, but that claims fail quietly, and only a discipline that turns each claim into tested evidence catches the failure while it is still cheap to catch.

Where people go wrong

  • "The vendor is a big, reputable company, so the data is fine." Reputation is not evidence. The Roomba images passed through an established, well-funded labeling vendor and still leaked (MIT Technology Review, 2022). Size changes the vendor's incentives and resources; it does not change your accountability or exempt any claim from verification. Verify the reputable vendor with the same rows as the unknown one.
  • "We have a contract, so we are covered." A contract allocates remedy; it does not clean the data. If non-consented data trains your model, a warranty clause gives you someone to sue, not a defensible system and not an unharmed data subject. Contract the gap, but never mistake a clause for verification. The clause is a response to residual risk, not a substitute for checking.
  • "De-identified and anonymized mean the same thing." They do not, and vendors exploit the blur. Anonymized data cannot reasonably be linked to a person; de-identified data has had direct identifiers stripped but may be re-identifiable. Precise location data sold as "anonymous" is routinely re-identifiable, which is why regulators have acted on it (FTC, 2024). Always ask which standard, by whose test.
  • "Consent exists, so consent covers us." Consent is purpose-bound. Data collected and consented for one use does not carry permission for a different one; that is the substance of consent archaeology (see Topic 2.2). A vendor's "consented" may be true for the original collection and false for your intended training use. Verify that the consent reaches your purpose, not just that some consent exists.
  • "Verification is the legal team's job, not governance's." Legal drafts the clause; governance decides whether the claim is true and what the residual risk is worth. The Analyze work of decomposing a claim into evidence is a governance skill, and it happens before the contract is written, not after. Handing the whole thing to legal produces clauses about risks no one measured.
  • "If we cannot verify it, we have to reject the deal." Not always. Rejection is one of four responses, alongside accept-and-document, contract-the-gap, and escrow-the-evidence. The governor prices the residual risk against the consequence and chooses proportionally. Reflexive rejection wastes good data; reflexive acceptance ships bad data. The skill is the middle.
  • "Sampling a public dataset is overkill." The depth of verification should match the consequence, but the assumption that public or cheap equals safe is how poisoned datasets spread. A public dataset behind a widely used model was found to contain illegal material (see Topic 2.1). Match effort to stakes, but do not confuse "free" with "verified."
  • "The labor that labels the data is an implementation detail." The labor link is a sub-processor with real access to your data and is where the Roomba leak occurred. "Our vendor handles labeling" hides a chain of people and devices you remain accountable for. Name the chain, control what reaches it, and treat "confidential workforce" as a red flag, not a reassurance.
  • "A cleaner, enriched dataset is a safer dataset." Enrichment makes data look better while it can quietly make the data riskier: joining two sources merges their consent problems, and appending an inferred sensitive attribute creates sensitive data out of ordinary inputs. Ask whether the product is a single source or a join, and what was inferred; a nicer file is not a cleaner provenance.
  • "If everyone in our field uses this dataset, it must be fine." Wide adoption spreads a defect rather than curing it. A public dataset behind a widely used model was found to contain illegal material precisely because adopters trusted its popularity instead of auditing it (see Topic 2.1). Popularity verifies popularity. Match verification to the consequence of your use, not to how many others skipped it.
  • "A pretrained model's training data is the provider's problem, not ours." You inherit the consequences of a model's training data when you deploy it, even though you cannot see the corpus. The defensible move is to read the disclosures that exist, check the litigation record, and record the undisclosed remainder as a named residual risk, not to write "third-party model, assumed clean." Invisible does not mean absent.

Questions people ask

What is third-party data?
Data whose original collection your organization did not control and whose history it did not witness, obtained by purchase, license, a data-broker feed, a data-labeling service, or as the training data inside a pretrained model. The defining property is that you inherit the data and its risk without having seen how it was collected, labeled, stored, or moved.
What is vendor claim?
An assertion a data seller makes about its data (for example "fully consented," "de-identified," "ethically sourced," "GDPR ready," "secure," "representative"). In governance, a claim is a hypothesis to be tested against evidence, not a fact to be filed, because it is written to close a deal rather than to survive an audit.
What is controller and processor?
Under the European Union's General Data Protection Regulation (GDPR), the controller decides why and how personal data is processed and carries the primary accountability; a processor acts on the controller's behalf. GDPR Article 28 requires the controller to use only processors offering sufficient guarantees, under a written contract, and requires processor authorization before engaging a sub-processor.
What is sub-processor?
A party a processor engages to help process data on the controller's behalf (for example a labeling subcontractor a data vendor uses). The controller must authorize sub-processors, which requires knowing they exist, so an unnamed or unauditable sub-processor is both a legal gap and a governance red flag. More on Sub-processor
What is verify?
To obtain and inspect the specific evidence that would prove a claim true or false, rather than to decide whether you believe the party making the claim. A claim is verified only when its proving evidence is in your hands; otherwise it is partially verified or unverifiable.

Keep going

Watch the lesson and prove it

Proving it happens inside the program: watch, explain it in your own words, and it goes on a record employers can check.

Watch the lesson and prove it →