Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(airtable): rename columns that contain colons #1218

Merged
merged 1 commit into from
Mar 14, 2022

Conversation

lauriemerrell
Copy link
Contributor

Overall Description

Manually rename two Airtable columns that were causing the Airtable operator to fail.

Led to creation of:

However since there was this lower-impact fix, I figured I'd just do this to unblock the DAG and we can prioritize the bigger-picture things longer term.

Checklist for all PRs

  • Run pre-commit run --all-files to make sure markdown/lint passes
  • Link this pull request to all issues that it will close using keywords (see GitHub docs about Linking a pull request to an issue using a keyword). Also mention any issues that are partially addressed or are related.

Airflow DAG changes checklist

  • Include this section whenever any change to a DAG in the airflow/dags folder occurs, otherwise please omit this section.

  • Verify that all affected DAG tasks were able to run in a local environment

  • Take a screenshot of the graph view of the affected DAG in the local environment showing that all affected DAG tasks completed successfully
    image

  • Add/update documentation in the docs/airflow folder as needed

  • Fill out the following section describing what DAG tasks were added/updated

This PR updates the airtable_loader DAG in order to updates the following DAG tasks:

  • california_transit_services: Add manual name overrides to handle column names containing colons

@lauriemerrell lauriemerrell added the airtable Items related to pulling data from Cal-ITP's airtable database. Evan Siroky is product owner. label Mar 14, 2022
@lauriemerrell lauriemerrell self-assigned this Mar 14, 2022
Copy link
Contributor

@atvaccaro atvaccaro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like to_snakecase should replace any non-alphanumerics instead of this hard-coded list...

@lauriemerrell lauriemerrell merged commit c5ddf49 into main Mar 14, 2022
@lauriemerrell lauriemerrell deleted the fix-airtable-graas-field-name branch March 14, 2022 15:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
airtable Items related to pulling data from Cal-ITP's airtable database. Evan Siroky is product owner.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants