From 0d29e0bbefedaa4af50c886dac9a1ddbf6c8077f Mon Sep 17 00:00:00 2001 From: Tatiana Al-Chueyr Date: Fri, 27 Dec 2024 08:04:43 +0000 Subject: [PATCH 1/2] Improve OpenLineage documentation Support users who may have questions similar to: * #1423 * #1430 --- docs/configuration/lineage.rst | 24 ++++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/docs/configuration/lineage.rst b/docs/configuration/lineage.rst index 8dfcae51b..bb18dcc16 100644 --- a/docs/configuration/lineage.rst +++ b/docs/configuration/lineage.rst @@ -4,8 +4,7 @@ Configuring Lineage =================== Since Cosmos 1.1, it uses internally the `openlineage-integration-common `_ -to parse artifacts generated with dbt commands and create lineage events. This feature is only available for the local -and virtualenv execution methods (read `execution modes <../getting_started/execution-modes.html>`_ for more information). +to parse artefacts generated with dbt commands and create lineage events. To emit lineage events, Cosmos can use one of the following: @@ -15,6 +14,27 @@ To emit lineage events, Cosmos can use one of the following: No change to the user DAG files is required to use OpenLineage. +Known limitations +----------------- + +This feature is only available for the local +and virtualenv execution methods (read `execution modes <../getting_started/execution-modes.html>`_ for more information). + +Additionally, since Cosmos uses the open-source `openlineage-integration-common `_, it relies on this library to support specific dbt adapters. As of 27 December 2024, the version 1.26.0 of this package supports: + +* Athena +* BigQuery +* Databricks +* DuckDB +* Dremio +* Postgres +* Redshift +* Snowflake +* Spark +* SQLServer + +Contributions are also welcome in the `OpenLineage project `_ to support more adaptors. + Installation ------------ From 296694a606f245224fd559d3fc75bc5c13163efd Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Fri, 27 Dec 2024 08:05:21 +0000 Subject: [PATCH 2/2] =?UTF-8?q?=F0=9F=8E=A8=20[pre-commit.ci]=20Auto=20for?= =?UTF-8?q?mat=20from=20pre-commit.com=20hooks?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/configuration/lineage.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/configuration/lineage.rst b/docs/configuration/lineage.rst index bb18dcc16..0dc6eb161 100644 --- a/docs/configuration/lineage.rst +++ b/docs/configuration/lineage.rst @@ -33,7 +33,7 @@ Additionally, since Cosmos uses the open-source `openlineage-integration-common * Spark * SQLServer -Contributions are also welcome in the `OpenLineage project `_ to support more adaptors. +Contributions are also welcome in the `OpenLineage project `_ to support more adaptors. Installation ------------