Dark data
A data source (an old warehouse table, a cached export, a spreadsheet a departed employee built) that feeds a pipeline without ever appearing in a formal source inventory or lineage map, because no one thought to register it. A chapter drafted only from named, registered sources will miss dark data by construction; the defensible response is a stated discovery method and date, not a claim that no dark data exists.
Defined in 2 GAGE programs, which carry 3 distinct definitions of it. The wording above is taught in Certified AI Data Governance Professional (CADGP).
How each discipline defines it
The same term does different work depending on who is using it. These are the definitions as each program teaches them, unedited.
Data an organization collects, generates, or stores that is never actively used, analyzed, or, most relevantly for this topic, monitored by anyone with clear accountability for it. Industry estimates of its share of total enterprise data vary widely by study and methodology; several studies over the past several years have suggested it commonly makes up a large share, in some cases a majority, of what organizations store, though any single percentage should be treated as illustrative rather than a fixed benchmark. A first-pass estate survey typically undercounts it because un-owned data is disproportionately data nobody is looking at.
Data that exists somewhere in an organization but is not on any current inventory, such as a forgotten export, an old shared-drive file, or a copy piped into an unapproved third-party tool. It cannot be scheduled for destruction until it is found, which is why an audit is a floor, not a ceiling.
A data source (an old warehouse table, a cached export, a spreadsheet a departed employee built) that feeds a pipeline without ever appearing in a formal source inventory or lineage map, because no one thought to register it. A chapter drafted only from named, registered sources will miss dark data by construction; the defensible response is a stated discovery method and date, not a claim that no dark data exists.
Where it is taught
The exact lessons this term appears in. The first module of every program is free with a free account.
- The un-owned dataset: finding data nobody claims and deciding who owns it now · The Estate Survey, Certified AI Data Governance Professional (CADGP)
- Data governance for the conformity file: your chapter in the document the regulator reads · Lineage Under Audit, Certified AI Data Governance Professional (CADGP)
- The retention decision: what you must keep, what you must destroy, and proving both · Data Reality, Certified AI Governance Professional (CAIGP)
Where it shows up at work
A definition is where Dark data starts. The lessons above teach it inside the competencies below, which are what job descriptions ask for and what the GAGE credential records. Every link here is drawn from the same careers map that builds a role's readiness check.
Collects, labels and preserves the evidence that a control operated, a decision was made, and a claim can be defended to an auditor or regulator.
One lesson on this page teach it as a primary outcome.
Explains training, tokens, context windows, embeddings, retrieval and fine-tuning well enough to ask an engineer a precise question and spot weak evidence.
One lesson on this page teach it as a primary outcome.
Traces where data came from, what transformed it, who owns each hop and where it flows downstream, so a number can be defended.
One lesson on this page teach it as a primary outcome.
Roles that hire for it
Each role page shows the full competency set, what the seat decides, and a readiness check against your own evidence.
- AI Auditorneeds it at depth · 9 open postings this week
- AI Evaluation Specialistneeds it at depth · 6 open postings this week
- AI Governance Engineerneeds it at depth · 7 open postings this week
- AI Model Validatorneeds it at depth
- Chief Audit Executive, AI audit editionneeds it at depth
- Senior AI Compliance Analystneeds working knowledge
How job descriptions say it
Postings rarely use the textbook word. These are the phrases the careers map reads as asking for the competencies above.
Where the frameworks place it: EU AI Act Articles 11 and 12; ISO/IEC 42001 Clause 7.5; EU AI Act Article 10; NIST AI RMF MAP 4.
One interview question about it
What evidence would you have ready before an auditor asks about an AI system, and how do you produce it as a byproduct of the work?
Asked of candidates for evidence collection and audit-ready documentation. Answering it in your own words is exactly how a GAGE topic is passed, and the credential records the pass where an employer can check it.
If you come from another field
Some backgrounds already carry this competency, and the readiness check gives credit for it.
- Compliance and regulatory affairs carries it as work you can show: Audit-ready documentation is what you produce.
- Internal audit and IT audit carries it as work you can show: Evidence evaluation is the profession.
- Project and program management carries it as a claim to back up: Decision logs and repositories are familiar.
Terms it appears with
Not an alphabetical neighbourhood: these are the terms taught in the same lessons, ranked by how often they appear together.