CI: drop python 3.9, numpy 1.21

[ev-br]

closes gh-230

This likely affects scikit-learn, scikit-learn/scikit-learn#30895 (comment)

[lucascolley]

scikit-learn have now dropped 3.9 and 1.21

[lucascolley]

Do we test with NuPy 1.22?

[lucascolley]

No 3.10?

[ev-br]

Yes, as an attempt to trim the CI size a bit. Experience over the last couple of months is that there were no issues which were python version dependent.

[ev-br]

Am planning to merge this next week if this looks reasonable to people. A possible contention point is the size of the CI matrix: this PR proposes to drop python 3.9 and 3.10 and numpy 1.21; this way we'll test on python 3.11, 3.12 and 3.13 x numpy 1.26, "latest released", and numpy-dev.

Jax, ndonnx, dask and pytorch: no changes, we only test with the latest released version.

[rgommers]

The linked scikit-learn issue has a clear plan, which is nice. It just dropped Python 3.9, so dropping Python 3.10 here seems quite aggressive - and probably not necessary? If you want to reduce the CI matrix, just remove 3.11 instead while keeping the lowest-supported version.

[lucascolley]

Similarly, I think we should be testing NumPy 1.22 as the min. supported by sklearn.

[lucascolley]

I spoke to Olivier and Tim yesterday and suggested that there shouldn't be any problems with array-api-compat and array-api-extra dropping more aggressively than scikit-learn, now that they have written down some reasonable rules. This matters to them as we are very close to them vendoring both libraries: scikit-learn/scikit-learn#30340

[lucascolley]

For comparison, in array-api-extra we define the following test envs

tests-numpy1 = ["py310", "tests", "numpy1"]
tests-py310 = ["py310", "tests"]
tests-py313 = ["py313", "tests"]
tests-backends = ["py310", "tests", "backends"]
tests-cuda = ["py310", "tests", "backends", "cuda-backends"]

and use the non-CUDA envs in the CI matrix

matrix:
    environment: [tests-py310, tests-py313, tests-numpy1, tests-backends]

Our numpy1 env is currently using oldest SPEC 0 NumPy (1.25), but I think we should change that to the oldest supported by sklearn (1.22) (data-apis/array-api-extra#169 (comment))

[lucascolley]

to clarify, I would like to see NumPy 1.22 tested before merging

[ev-br]

Well, from past experience there was not much useful signal from keeping both numpy 1.21 and 1.26. All it did was a bit was a bit more churn to duplicate xfails for the two numpy versions. If we are to add 1.22, we can keep 1.21 just as well.

[lucascolley]

How about 1.22 instead of 1.26?

[ev-br]

Honestly, I don't see much point.
Esp given that we're planning to make parts of the testing matrix only compatible with python 3.12+ for _buffer__,
https://hackmd.io/zn5bvdZTQIeJmb3RW1B-8g#Downstream-libraries-update, and numpy 1.26 seems to be the first one to even support python 3.12.
Granted, it's mainly for array-api-strict, not array-api-compat. So technically, we can keep -compat very very conservative. Why though.

[lucascolley]

Well, I suppose we should ask the scikit-learn devs, as they are the 'users' of the NumPy 1.22 support. Does you foresee it being useful to catch things in array-api-compat CI before they propagate to array-api-extra or scikit-learn, @betatim?

[crusaderky]

My 2 cents on NumPy 1.22 is that

1. it's out of SPEC0 so it's reasonable to ask scikit-learn developers to drop it. I think it would also be reasonable to say that they won't be able to upgrade array-api-compat beyond 1.11 until they do so.
2. however, if it is supported, it must be tested.

FYI SPEC0 dropped NumPy 1.25 two days ago.

For the record, I do not advocate being this much aggressive when SPEC0 will drop 1.26 three months from now. But asking final users to upgrade to the latest available 1.x should be painless for most.

I just verified that NumPy 1.22 does have _CopyMode and would work without this special casing, whereas 1.21 doesn't:

array-api-compat/array_api_compat/numpy/_aliases.py

Lines 101 to 112 in b6900df

	if hasattr(np, '_CopyMode'):
	if copy is None:
	copy = np._CopyMode.IF_NEEDED
	elif copy is False:
	copy = np._CopyMode.NEVER
	elif copy is True:
	copy = np._CopyMode.ALWAYS
	else:
	# Not present in older NumPys. In this case, we cannot really support
	# copy=False.
	if copy is False:
	raise NotImplementedError("asarray(copy=False) requires a newer version of NumPy.")

that said, I don't know (without running the unit tests) if anything else would still require a special code path.

[crusaderky]

Suggested change

	python-version: ['3.10', '3.12']
	python-version: ['3.10', '3.12', '3.13']

[lucascolley]

3.12 testing can be dropped from CI?

[crusaderky]

not if you want to test numpy 1.26 on the most recent python version it supports.
But yes, we could run on 3.12 for numpy 1.x and on 3.13 for everything else.

[crusaderky]

Can't we just use an exclude: directive like in tests.yml below?

[lucascolley]

it's out of SPEC0 so it's reasonable to ask scikit-learn developers to drop it. I think it would also be reasonable to say that they won't be able to upgrade array-api-compat beyond 1.11 until they do so.

We have asked and the clear answer was that scikit-learn won't be following SPEC 0 in its current form. They have written down concrete rules for dropping versions at https://scikit-learn.org/dev/developers/maintainer.html#guideline-for-bumping-minimum-versions-of-our-dependencies. I hope that SPEC 0 can be adjusted so that scikit-learn is compliant (perhaps at the Scientific Python summit this year), but I think preventing scikit-learn from using the latest versions of these libraries has the potential to be very problematic, for insignificant practical gain.

[crusaderky]

https://scikit-learn.org/dev/developers/maintainer.html#guideline-for-bumping-minimum-versions-of-our-dependencies

Forget about 1.22 vs. 1.26. This policy implies that they commit to support NumPy 1.x until October 2027, when Python 3.12 will fall out of their 4 years window. By then, SPEC0 will recommend NumPy >=2.3.
This is a lot of luggage that everyone else who maintains their dependencies is asked to carry too.

NEP29 first and SPEC0 then were written for a reason. I find it a bit baffling - not that one library can decide to diverge (to each the problems they choose for themselves) - but that other libraries, whose developers likely disagree with their decision, are asked to accommodate them.

[lucascolley]

I hope that SPEC 0 can be adjusted so that scikit-learn is compliant

support NumPy 1.x until October 2027

perhaps I am hoping for more of a compromise then :)

[betatim]

I'm not sure I can add much here (sorry for being a few days late Lucas). scikit-learn will follow its policy for the next while. Both because it has been its inofficial policy for a long time and because getting agreement on making it official policy took some effort. Of course in the future things will get reconsidered and we left an escape hatch to drop things earlier if there is a good reason. Right now I think it will be hard to convince the rest of scikit-learn to accelerate the Numpy schedule because of a compat library that is used by an experimental feature of scikit-learn (at least I'm hard to convince).

I think having a CI job that tests the oldest set of dependencies is worth having, and skipping intermediate versions is a good way to reduce CI matrix size.