Skip to content

Fix BigQuery private-cloud trigger result payloads#1

Draft
cursor[bot] wants to merge 2 commits intomainfrom
cursor/critical-bug-inspection-cbe5
Draft

Fix BigQuery private-cloud trigger result payloads#1
cursor[bot] wants to merge 2 commits intomainfrom
cursor/critical-bug-inspection-cbe5

Conversation

@cursor
Copy link
Copy Markdown

@cursor cursor Bot commented May 7, 2026

Fix private-cloud BigQuery deferrable trigger result payloads.

Bug and impact:

  • In non-default Google Cloud universes, BigQueryGetDataTrigger and BigQueryIntervalCheckTrigger use the synchronous BigQuery hook to fetch query results.
  • That hook returns dict rows containing native Python values such as Decimal and date.
  • BigQueryGetDataTrigger returned those dictionaries directly even when as_dict=False, breaking the documented list-of-lists contract, and both triggers could emit values that fail triggerer IPC serialization after the external BigQuery job had already completed.

Root cause:

  • The private-cloud path bypassed the async REST result conversion used by the default-universe path, so it did not normalize row shape or convert native BigQuery client values into trigger-safe values.

Fix:

  • Normalize private-cloud synchronous BigQuery rows in the trigger before yielding TriggerEvent payloads.
  • Preserve as_dict=True vs as_dict=False output shape for BigQueryGetDataTrigger.
  • Convert non-primitive result values to trigger-safe representations, including base64 for bytes and ISO strings for date/time values.
  • Cover the regression with focused unit tests using Decimal and date row values.

Validation:

  • uvx ruff format providers/google/src/airflow/providers/google/cloud/triggers/bigquery.py providers/google/tests/unit/google/cloud/triggers/test_bigquery.py
  • uvx ruff check --fix providers/google/src/airflow/providers/google/cloud/triggers/bigquery.py providers/google/tests/unit/google/cloud/triggers/test_bigquery.py
  • uv run --project providers/google --no-dev --with pytest --with pytest-asyncio --with-editable ./devel-common pytest providers/google/tests/unit/google/cloud/triggers/test_bigquery.py::TestBigQueryGetDataTrigger::test_bigquery_get_data_trigger_success_with_data_custom_universe providers/google/tests/unit/google/cloud/triggers/test_bigquery.py::TestBigQueryIntervalCheckTrigger::test_interval_check_trigger_success_non_default_universe -xvs (3 passed)

Notes:

  • The standard uv run --project providers/google pytest ... setup failed in this image before tests because provider dev extras attempted to build mysqlclient without MySQL/MariaDB pkg-config libraries.
  • The Breeze fallback could not run because Docker is not installed in this automation image.

Was generative AI tooling used to co-author this PR?
  • Yes — GPT-5.5

Generated-by: GPT-5.5 following the guidelines


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
Open in Web View Automation 

cursoragent and others added 2 commits May 7, 2026 11:17
Co-authored-by: csrinivas2130 <csrinivas2130@users.noreply.github.com>
Co-authored-by: csrinivas2130 <csrinivas2130@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant