AI Governance Engineer
GitHub

Provenance and lineage

Provenance records where a dataset came from and on what terms; lineage traces it backward from a model to its sources and forward to every model using it.

By Updated First published

Provenance and lineage Provenance records where a dataset came from and on what terms; lineage traces it through pipeline jobs and runs, backward from a model to its sources and forward from a dataset to every model that used it. Where data came from, where it went Source licence, rights Provenance where from, on what terms Dataset datasheet Pipeline jobs and runs Model A Model B Backward what fed this model? Forward which models used this dataset? An erasure request or a licence withdrawal asks the forward question. Granularity, by where rights attach Dataset level: the default Record level: rights attach to records Feature level: sensitive derived proxies Lineage events about datasets, jobs and runs
Provenance and lineage Provenance says where a dataset came from and on what terms; lineage traces how it moved through pipeline jobs and runs, backward from a model and forward from a dataset. Record lineage forward as well as back, because an erasure request or a licence withdrawal asks which models used the data. Drawn from chapter 14.

Text alternative

Provenance is where a dataset came from and on what terms. Lineage is how the data moved and changed through pipelines, recorded as lineage events about datasets, jobs and runs. Lineage runs both ways: backward lineage answers "what fed this model?", forward lineage answers "which models used this dataset?", and the second question is the one an erasure request or a licence withdrawal asks. Granularity follows where rights attach: dataset-level provenance is the default, record-level provenance is needed where rights attach to records (personal data, per-source licences, opt-outs), and feature-level lineage is needed for sensitive derived features that can act as proxies. The human-readable companion is a datasheet.

Download

Every file carries the attribution band "aigovernanceengineer.com · CC BY 4.0 · v0.5.0" inside the image, and the version is in the file name, so a copy always says where it came from and which edition it shows. The SVGs keep the text live: the first follows the viewer's light or dark setting, the other two fix one theme for slides and print. The PNGs are drawn with the site's own typefaces.

Provenance records where a dataset came from and on what terms; lineage traces it backward from a model to its sources and forward to every model using it.
The PNG, light, 1600 px wide, as it downloads (shown here as a lighter copy).

Reuse and credit

The figure is published under CC BY 4.0: you may copy, share and adapt it, commercially too, provided you give appropriate credit, link to the licence and say if you changed it. Keep the attribution band in the image. A credit line that covers title, author, source and licence:

“Provenance and lineage” by Jorge García Aibar, aigovernanceengineer.com (https://aigovernanceengineer.com/figures/provenance-lineage), v0.5.0. Licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/).

Embed with HTML

<figure>
  <img src="https://aigovernanceengineer.com/downloads/figures/provenance-lineage-v0.5.0-light-1600.png" alt="Provenance records where a dataset came from and on what terms; lineage traces it backward from a model to its sources and forward to every model using it." width="800" height="1088" loading="lazy">
  <figcaption>
    <a href="https://aigovernanceengineer.com/figures/provenance-lineage">Provenance and lineage</a> by Jorge García Aibar,
    aigovernanceengineer.com, v0.5.0.
    Licensed under <a href="https://creativecommons.org/licenses/by/4.0/">CC BY 4.0</a>.
  </figcaption>
</figure>

Embed with Markdown

![Provenance records where a dataset came from and on what terms; lineage traces it backward from a model to its sources and forward to every model using it.](https://aigovernanceengineer.com/downloads/figures/provenance-lineage-v0.5.0-light-1600.png)

*[Provenance and lineage](https://aigovernanceengineer.com/figures/provenance-lineage) by Jorge García Aibar, aigovernanceengineer.com, v0.5.0. Licensed under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/).*