False positives caused by strict equality comparison on NumPy 1.22?

[crusaderky]

I ran the tests on 100k examples (>8h runtime) on array_api_compat.numpy on NumPy 1.22 (xfails from data-apis/array-api-compat#304), and got several failures.
I didn't investigate thoroughly yet but I suspect that at least some of them may be related to using strict equality comparison instead of almost_eq with a very low relative tolerance.
The equality test is done by _pytest_helpers.py by calling numpy.ndarray.__eq__, which I suspect had issues in old versions of NumPy?

FAILED array_api_tests/test_creation_functions.py::test_linspace - AssertionError: out[0]=0.0, but should be -0.0 [linspace()]
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_clip - AssertionError: out[()]=-inf but should be -3.4028235677973366e+38 [clip()]
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_equal[equal(x1, x2)] - AssertionError: out[0]=True, but should be (x1 == x2[0])=False [equal()]
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_equal[__eq__(x1, x2)] - exceptiongroup.ExceptionGroup: Hypothesis found 2 distinct failures. (2 sub-exceptions)
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_greater[greater(x1, x2)] - AssertionError: out[0]=False, but should be (x1 > x2[0])=True [greater()]
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_greater[__gt__(x1, x2)] - exceptiongroup.ExceptionGroup: Hypothesis found 2 distinct failures. (2 sub-exceptions)
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_greater_equal[greater_equal(x1, x2)] - AssertionError: out[0]=True, but should be (x1 >= x2[0])=False [greater_equal()]
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_greater_equal[__ge__(x1, x2)] - AssertionError: out[0]=True, but should be (x1 >= x2[0])=False [__ge__()]
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_less[less(x1, x2)] - exceptiongroup.ExceptionGroup: Hypothesis found 2 distinct failures. (2 sub-exceptions)
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_less[__lt__(x1, x2)] - AssertionError: out[0]=False, but should be (x1 < x2[0])=True [__lt__()]
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_less_equal[less_equal(x1, x2)] - AssertionError: out[0]=True, but should be (x1 <= x2[0])=False [less_equal()]
FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_less_equal[__le__(x1, x2)] - exceptiongroup.ExceptionGroup: Hypothesis found 2 distinct failures. (2 sub-exceptions)
FAILED array_api_tests/test_searching_functions.py::test_where - AssertionError: out[(0, 0, 0, 0)]=16777216j, but should be _x2[(0, 0, 0, 0)]=16777217j [where()]

[crusaderky]

NVM, most of these were in the previous xfails file; they just don't show up until you run with an enormous amount of examples

[ev-br]

I ran the tests on 100k examples (>8h runtime)

Wow :-).

So the fix is to not remove these xfails (were you removing them in one of array-api-compat PRs?), maybe add a comment to this effect.

[crusaderky]

I'm a bit conflicted about this one

FAILED array_api_tests/test_creation_functions.py::test_linspace - AssertionError: out[0]=0.0, but should be -0.0 [linspace()]

This is caused by linspace(-0.0, -1.0, num=1) returning [+0.0] instead of [-0.0].

while I sort of understand how some user function may treat -0. and +0. as different, I can't find any mention in the Array API that the two numbers must be treated as different, so I am unsure if numpy 1.22 is failing foul of the Standard?