Skip to main content
Version: 96.1

Lineage

Introduction​

Factor Platform captures lineage metadata across your connected source systems (such as Flink and Kafka) to provide a clear view of data flow and dependencies across your environment.

Factor Platform uses the OpenLineage standard to describe its data lineage. The OpenLineage specification provides a vendor-neutral model for representing datasets, jobs, and runs, making it possible to integrate lineage data across your other tools and systems.

Right now Factor Platform's support for lineage metadata is limited to Schema Registry datasets. This metadata enriches the information available for each schema resource. For example, identifying whether a schema contains PII data or filtering schemas by owner.

In future releases, Factor Platform will also:

  • Produce OpenLineage events, allowing you to feed lineage metadata into other systems in your data governance stack.
  • Consume OpenLineage events from other producers in your stack (such as Flink jobs and Iceberg tables)

Getting started​

To get started with lineage, see the Sources section of our documentation. Sources are the data systems that Factor Platform extracts lineage metadata from - configuring a source is the first step to populating lineage data in Factor Platform.