Pathspec max version of ~=0.9.0?

[millsks]

We use conda as our package manager. We install apache-airflow, but we are also looking at installing DVC too. When trying to install the DVC package into our environment that also includes apache-airflow the DVC package requires pathspec >=0.10.3 so it fails to solve the required environment specs because of airflow's current restriction of pathspec >=0.9.0,<0.10.0. At most we can install DVC 2.39.0 because of this restriction.

My question being was there a reason to restrict pathspec ~=0.9.0?

Could not solve for environment specs
The following packages are incompatible
├─ apache-airflow >=2.4.0  is installable and it requires
│  └─ airflow [>=2.4.0,<2.4.1.0a0 |>=2.4.1,<2.4.2.0a0 |...|>=2.6.3,<2.6.4.0a0 ], which requires
│     └─ pathspec [>=0.9.0,<0.10.0 |>=0.9.0,<0.10.dev0 ], which can be installed;
└─ dvc >=3.14.0  is uninstallable because it requires
   └─ pathspec >=0.10.3 , which conflicts with any installable versions previously reported.

[potiuk]

Not sure. It's been limited before we introduced the rule where we describe the limitations. But feel free to make a PR to attempt to change it - and see if any tests fail, if not then likely reason for limiting is gone and we will be able to remove it.

[potiuk]

I think it was just overcautious - it's only about glob ignore rules in .airflowignore - so removing the limit (or better - changing to >=0.9.0 should be perfectly fine once the tests pass)

[millsks]

Thanks. I had set up the airflow dev environment locally as suggested in CONTRIBUTORS_QUICK_START. If I was following along correctly the first step would be to update pathspec>=0.9.0 in setup.cfg and then run breeze ci-image build --upgrade-to-newer-dependencies to rebuild the image with the updated dependency requirements? I will get an issue opened up for this, but its not really a bug so is there a specific issue type I should open up to help categorize the request better? If possible I would like to help out with this where I can. I don't mind doing the all the grunt work as it is helping me to learn more about airflow :)

[millsks]

What is the average build time when updating a dependency? I noticed the backtracking messages and the current build has been running for about 25 minutes.

[eladkal]

What is the average build time when updating a dependency?

Since this is your first PR to the repo any commit nerds to be approved by mantainers before CI starts.

I approved the run. Lets see the tests results.

[millsks]

Thanks @eladkal! I submitted #33349 and related it to this discussion.