dependencies.md

DBT sources and upstream dependencies

DBT model blocks can depend on other DBT models and other blocks: sensors, Python, SQL, R.

DBT models depending on other DBT models

When you add a DBT model block to your pipeline, if it contains any refs, those DBT models will be added to the current pipeline as upstream dependencies to the DBT model you initially added.

DBT models depending on other blocks

When a DBT model depends on a block that isn’t another DBT model, a YAML file is created named mage_sources.yml. This file is automatically generated and updated whenever a new non-DBT model block is an upstream dependency of a DBT model block.

If you have a Mage project named demo_project with a DBT project named demo, here is an example of where the mage_sources.yml file could be located:

demo_project/
|   dbt/
|   |   demo/
|   |   |   models/
|   |   |   |   example/
|   |   |   |   |   mage_sources.yml
|   |   |   |   |   my_first_dbt_model.sql
|   |   |   |   |   my_second_dbt_model.sql

Sources

The contents of the mage_sources.yml could look like this:

sources:
- name: mage_demo_project
  tables:
  - name: demo_pipeline_load_data
version: 2

The source name follows this convention: mage_[Mage_project_name].

The table names under the source follow this convention: [pipeline_uuid]_[block_uuid].

For more information on sources, read DBT’s documentation.