How to backfill one or several tasks in a dynamic task group?

[da-tubi]

There is a dag with a dynamic task group containing:

• task1
• task2
• ...
• taskN

Now, we need to backfill only some of them. We can not directly backfill them by the UI because there are tasks before the task group. And the tasks before the task group must be executed only and exactly once. The problem is:

• How to backfill only one task in the task group?
• How to pick several tasks to backfill in the task group?
• How to backfill one/several tasks in a date range?

Here is my answer to the question: https://github.com/da-tubi/pants-airflow
Hope to find a better one!

[potiuk]

If I were you, I'd rather use Airflow API where the built-in features of Airlfow are not enough. Wriitng Python and using API is the first-class-citizen of Airflow usage. Airflow is really (especially with 2.0 and beyond) a programmable platform, that can be extended and USING API for that is likely a good idea.