Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix pip upgrades in dbt examples #24001

Merged
merged 1 commit into from
Aug 28, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ Our example updates the CI/CD files of a project from a GitHub repository, but t
```yaml
- name: Prepare DBT project for deployment
run: |
pip install pip --upgrade
python -m pip install pip --upgrade
pip install . --upgrade --upgrade-strategy eager ## Install the Python dependencies from the setup.py file, ex: dbt-core and dbt-duckdb
dagster-dbt project prepare-and-package --file <DAGSTER_PROJECT_FOLDER>/project.py ## Replace with the project.py location in the Dagster project folder
shell: bash
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -195,7 +195,7 @@ The last step is to update the [CI/CD files](/dagster-plus/references/ci-cd-file
- name: Prepare DBT project for deployment
if: steps.prerun.outputs.result == 'pex-deploy'
run: |
pip install pip --upgrade
python -m pip install pip --upgrade
cd project-repo
pip install . --upgrade --upgrade-strategy eager ## Install the Python dependencies from the setup.py file, ex: dbt-core and dbt-duckdb
dagster-dbt project prepare-and-package --file <DAGSTER_PROJECT_FOLDER>/project.py ## Replace with the project.py location in the Dagster project folder
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,12 @@ To get our deployment working, we need to add a step to our GitHub Actions workf
2. Open the `deploy.yml` file.
3. Locate the `Checkout for Python Executable Deploy` step, which should be on or near line 38.
4. After this step, add the following:

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@erinkcochran87 do you know if these formatting changes are anything to worry about, is there an equivalent of make mdx-format for daggy u?

```yaml
- name: Prepare DBT project for deployment
if: steps.prerun.outputs.result == 'pex-deploy'
run: |
pip install pip --upgrade
python -m pip install pip --upgrade
cd project-repo
pip install . --upgrade --upgrade-strategy eager
dagster-dbt project prepare-and-package --file dagster_university/project.py
Expand All @@ -33,7 +33,7 @@ To get our deployment working, we need to add a step to our GitHub Actions workf
The code above:

1. Creates a step named `Prepare DBT project for deployement`
2. Upgrades `pip`, the package installer for Python
2. Upgrades `pip`, the package installer for Python
3. Navigates inside the `project-repo` folder
4. Upgrades the project dependencies
5. Prepares the manifest file by running the `dagster-dbt project prepare-and-package` command, specifying the file in which the `DbtProject` object is located.
Expand All @@ -58,9 +58,9 @@ At this point, your dbt project will be successfully deployed onto Dagster+ and

## Experiencing issues?

There are two ways to deploy Dagster+ Serverless. In our case, we only made changes to deploy dbt with the default option, called Fast Deploys with PEX (Python Executable).
There are two ways to deploy Dagster+ Serverless. In our case, we only made changes to deploy dbt with the default option, called Fast Deploys with PEX (Python Executable).

**If you receive an error message to turn off Fast Deploys**, GitHub Actions will skip the steps that build the dbt project. We recommend staying with Fast Deploys, if possible.
**If you receive an error message to turn off Fast Deploys**, GitHub Actions will skip the steps that build the dbt project. We recommend staying with Fast Deploys, if possible.

**If your project works locally**, the issue is likely a mismatch between where Dagster looks for the manifest locally and in production. This can be resolved by tinkering with where the manifest is being written.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ jobs:
- name: Prepare DBT project for deployment
if: steps.prerun.outputs.result == 'pex-deploy'
run: |
pip install pip --upgrade
python -m pip install pip --upgrade
cd project-repo
pip install . --upgrade --upgrade-strategy eager
dagster-dbt project prepare-and-package --file assets_dbt_python/project.py
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ jobs:
- name: Prepare DBT project for deployment
if: steps.prerun.outputs.result == 'pex-deploy'
run: |
pip install pip --upgrade
python -m pip install pip --upgrade
cd project-repo
pip install . --upgrade --upgrade-strategy eager
dagster-dbt project prepare-and-package --file assets_dbt_python/project.py
Expand Down