Data lineage tracking
Data lineage tracking traces where a piece of data came from and every transformation it passed through before reaching its current form.
What is data lineage tracking?
Data lineage tracking records the path a piece of data took from its original source, through every job, pipeline, or transformation that touched it, to wherever it ended up. It answers two questions: where did this number come from, and what would break if someone changed something upstream of it. Without lineage tracking, a number in a dashboard has no traceable origin, so a wrong value can't be tracked back to the step that produced it.
How does data lineage tracking differ from version history?
Lineage tracking follows data across systems: a source table, the pipeline that transforms it, and the report it lands in. Version history tracks the changes to one thing over time, keeping every prior state of a single file or record so you can see what it said before and after each edit. Lineage answers where a fact came from and what it passed through. Version history answers what a specific file looked like at any point in its own history. A complete picture of a piece of data usually needs both.
How does data lineage tracking work?
Every job or pipeline that reads or writes data emits metadata about what it consumed and what it produced, building a graph of datasets connected by the jobs that moved data between them. Tools built for this define a common format so different pipeline tools can report lineage events the same way instead of each system tracking it in its own incompatible format. Qontext's context repository ties every file back to the source it came from and keeps a version history of every change since, so a fact an agent retrieves stays traceable to the connected source it originated from, without a separate lineage tool.