Skip to content

Serialize dag if it is not serialized even if it does not have task instances#68947

Open
kgskgs wants to merge 1 commit into
apache:mainfrom
kgskgs:fix-serialize-skipped
Open

Serialize dag if it is not serialized even if it does not have task instances#68947
kgskgs wants to merge 1 commit into
apache:mainfrom
kgskgs:fix-serialize-skipped

Conversation

@kgskgs

@kgskgs kgskgs commented Jun 24, 2026

Copy link
Copy Markdown

Prevent dags from not ever being serialized if they have no entry in serialized_dag table, but have one in dag_version
closes: #68944


Context:
I ran into this state while running a custom application on top of airflow. It is highly likely that it was caused by custom code and not airflow, but airflow doesn't seem to have any safeguards against entering the state, and it affected all instances of my application. I can't easily tell exactly how it happened exactly (we are in the middle of updating from airflow 2 to 3 and there are lots of changes)
I believe this fix is appropriate given that there should be multiple ways to reach this state e.g. by editing the db manually.
An alternative would be to try and add safeguards against getting into this state in the first place, but that would be way more complex than handling in this way + currently this is done at no additional cost as we already know whether the serialized dag exists or not by the old hash.
There should be no implications on backwards compatibility.


Was generative AI tooling used to co-author this PR?
  • Yes (for the unit test)

Generated-by: Claude Sonnet 4.6 following the guidelines


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@boring-cyborg

boring-cyborg Bot commented Jun 24, 2026

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

@SameerMesiah97 SameerMesiah97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs more explanation. Please see my comment on the test.

session.execute(delete(SDM).where(SDM.dag_id == dag.dag_id))
session.flush()
assert session.scalar(select(func.count()).select_from(SDM)) == 0
assert session.scalar(select(func.count()).select_from(DagVersion)) == 1

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am just not sure how this state you are simulating can arise under normal operation. The test simulates the issue by manually deleting the SerializedDagModel row while leaving the DagVersion row intact. Since DAGs are serialized during parsing, I would expect DagVersion and SerializedDagModel to be created together under normal operation.

Could you expand the issue/PR description to explain how this state can occur in practice? Is this intended to recover from metadata corruption, failed migrations, manual database modification, or some known failure during serialization?

Furthermore, have you ran into this specific failure mode yourself?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I've ran into this myself, but I am not sure what caused it, I am also running a custom application built on top of airflow so most likely it was not caused by airflow itself (I suspect it happened when upgrading from airflow 2 to 3 but there is lots of custom code, sometimes too tightly coupled to airflow and it would be a lot of effort to determine exactly how it happened)
I'd say that there being nothing preventing you from getting into this state + the user being able to manually break their dags by editing the db is reason enough to have this

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I've ran into this myself, but I am not sure what caused it, I am also running a custom application built on top of airflow so most likely it was not caused by airflow itself (I suspect it happened when upgrading from airflow 2 to 3 but there is lots of custom code, sometimes too tightly coupled to airflow and it would be a lot of effort to determine exactly how it happened) I'd say that there being nothing preventing you from getting into this state + the user being able to manually break their dags by editing the db is reason enough to have this

I would add all the detail you can in both the issue and the PR description (please include the what, why and how as well as any implications for backwards compatibility).

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@eladkal eladkal requested a review from kaxil June 25, 2026 08:53
@eladkal eladkal added this to the Airflow 3.3.1 milestone Jun 25, 2026
@eladkal eladkal added type:bug-fix Changelog: Bug Fixes backport-to-v3-3-test Backport to v3-3-test labels Jun 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:DAG-processing backport-to-v3-3-test Backport to v3-3-test type:bug-fix Changelog: Bug Fixes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DAGs that have entires in dag_version but not in serialized_dag are never serialized

3 participants