GH-48473: [CI][Python] Require numpy 2.0 by alippai · Pull Request #50769 · apache/arrow

alippai · 2026-08-02T17:26:59Z

Rationale for this change

Require Numpy 2.0 to support newer features, allow newer numpy API/ABI

What changes are included in this PR?

Numpy 2.0+ requirement
Pandas 2.2.2+ requirement (first wheel compatible with numpy 2.0)
Pyodide 0.27.1 usage (numpy 2.0 api)

Are there any user-facing changes?

Yes, in contrast to pyarrow 25, after merging this PR pyarrow 26 can't be used with Numpy 1.x or pandas 2.0-2.2.1

This PR includes breaking changes to public APIs.

This PR was created using GPT-5.6-Sol-xhigh, every line read & reviewed by me.

GitHub Issue: [Python] Require numpy 2.x #48473

github-actions · 2026-08-02T17:27:24Z

⚠️ GitHub issue #48473 has been automatically assigned in GitHub to PR creator.

alippai · 2026-08-02T19:21:25Z

@tadeja @rok @raulcd @jorisvandenbossche this is ready for review. The failure seems to be an unrelated arrow flight timeout. Once this is on master, I'll file a new StringDType from/to conversion PR.

kou · 2026-08-03T05:20:54Z

@github-actions crossbow submit -g python

github-actions · 2026-08-03T05:25:25Z

Revision: d0a51cc

Submitted crossbow builds: ursacomputing/crossbow @ actions-764bbcb6f5

Task	Status
example-python-minimal-build-fedora-conda
example-python-minimal-build-ubuntu-venv
test-conda-python-3.11
test-conda-python-3.11-dask-latest
test-conda-python-3.11-dask-upstream_devel
test-conda-python-3.11-hdfs-2.9.2
test-conda-python-3.11-hdfs-3.2.1
test-conda-python-3.11-hypothesis
test-conda-python-3.11-pandas-2.2.2-numpy-2.0.2
test-conda-python-3.11-spark-master
test-conda-python-3.12
test-conda-python-3.12-pandas-latest-numpy-latest
test-conda-python-3.13
test-conda-python-3.13-pandas-latest-numpy-2.1.0
test-conda-python-3.13-pandas-latest-numpy-latest
test-conda-python-3.14
test-conda-python-3.14-cpython-debug
test-conda-python-3.14-pandas-nightly-numpy-nightly
test-conda-python-3.14-pandas-upstream_devel-numpy-nightly
test-conda-python-emscripten
test-debian-13-python-3-amd64
test-debian-13-python-3-i386
test-fedora-42-python-3
test-ubuntu-24.04-python-3

raulcd

Thanks @alippai for working on this. Changes LGTM in general, let me lauch extra CI and get the ok from @pitrou @jorisvandenbossche @AlenkaF

raulcd · 2026-08-03T09:08:25Z

            python: "3.11"
-            pandas: "2.0.3"
-            numpy: "1.23.2"
+            pandas: "2.2.2"


@jorisvandenbossche is the bump to pandas 2.2.2 ok from your perspective in order to support numpy > 2?

That should be fine I think (well, it is also unavoidable if we want to only support numpy >= 2)

raulcd · 2026-08-03T09:13:07Z

@github-actions crossbow submit wheel-cp313

github-actions · 2026-08-03T09:19:50Z

Revision: d0a51cc

Submitted crossbow builds: ursacomputing/crossbow @ actions-449ef0ee7b

Task	Status
wheel-macos-monterey-cp313-cp313-amd64
wheel-macos-monterey-cp313-cp313-arm64
wheel-manylinux-2-28-cp313-cp313-amd64
wheel-manylinux-2-28-cp313-cp313-arm64
wheel-musllinux-1-2-cp313-cp313-amd64
wheel-musllinux-1-2-cp313-cp313-arm64
wheel-windows-cp313-cp313-amd64

raulcd · 2026-08-03T10:11:55Z

@github-actions crossbow submit wheel-cp311

github-actions · 2026-08-03T10:14:47Z

Revision: d0a51cc

Submitted crossbow builds: ursacomputing/crossbow @ actions-012e7c747c

Task	Status
wheel-macos-monterey-cp311-cp311-amd64
wheel-macos-monterey-cp311-cp311-arm64
wheel-manylinux-2-28-cp311-cp311-amd64
wheel-manylinux-2-28-cp311-cp311-arm64
wheel-musllinux-1-2-cp311-cp311-amd64
wheel-musllinux-1-2-cp311-cp311-arm64
wheel-windows-cp311-cp311-amd64

rok

Thanks for working on this @alippai !

I found some other places where version checks occur 1, 2, 3, 4, 5. Could you take a look and see if we can simplify/remove those?

ngoldbaum

I asked an AI model to review this with an eye toward code that can be deleted to simplify interfacing with Pandas and it found several more places where you can delete or simplify code. I'm happy to take on these cleanups in a followup if you think my suggestions are scope creep.

alippai · 2026-08-03T16:26:48Z

@ngoldbaum @rok thanks, I’ll address the pandas upgrade too. It was addressing the numpy upgrade only. I believe pandas improvements are in scope for this MR. Addressing upstream fixes and even onboarding the optimal/newer APIs allowed by the new version should land together.

ngoldbaum

Just one last nit but this otherwise looks good to me!

…d docs.

jorisvandenbossche

Looks good to me!

jorisvandenbossche · 2026-08-06T13:14:06Z

            python: "3.11"
-            pandas: "2.0.3"
-            numpy: "1.23.2"
+            pandas: "2.2.2"


That should be fine I think (well, it is also unavoidable if we want to only support numpy >= 2)

jorisvandenbossche · 2026-08-06T13:17:49Z

+{% for python_version, pandas_version, numpy_version, cache_leaf in [("3.11", "2.2.2", "2.0.2", True),
                                                                     ("3.12", "latest", "latest", False),
-                                                                     ("3.13", "latest", "1.26.2", False),
+                                                                     ("3.13", "latest", "2.1.0", False),


Suggested change

("3.13", "latest", "2.1.0", False),

("3.13", "latest", "2.1.3", False),

(seems more logical to test with the latest bug-fix release from a certain series?)

alippai requested review from AlenkaF, assignUser, jonkeane, kou, pitrou, raulcd and rok as code owners August 2, 2026 17:27

github-actions Bot added Component: C++ Component: Python Component: Documentation awaiting review Awaiting review labels Aug 2, 2026

alippai force-pushed the alippai-numpy-20 branch 2 times, most recently from 459c7f0 to d0a51cc Compare August 2, 2026 18:42

raulcd reviewed Aug 3, 2026

View reviewed changes

github-actions Bot added awaiting changes Awaiting changes and removed awaiting review Awaiting review labels Aug 3, 2026

rok reviewed Aug 3, 2026

View reviewed changes

Comment thread python/pyarrow/tests/test_extension_type.py Outdated

ngoldbaum mentioned this pull request Aug 3, 2026

[Python] Conversion to/from numpy 2.0+ new StringDType #42018

Open

ngoldbaum reviewed Aug 3, 2026

View reviewed changes

alippai force-pushed the alippai-numpy-20 branch from cb5085f to d0a51cc Compare August 3, 2026 15:59

github-actions Bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Aug 3, 2026

alippai requested review from ngoldbaum and rok August 3, 2026 23:54

alippai force-pushed the alippai-numpy-20 branch from 5c4831f to dc6ef99 Compare August 4, 2026 00:20

ngoldbaum approved these changes Aug 4, 2026

View reviewed changes

Comment thread python/pyarrow/types.pxi Outdated

alippai added 7 commits August 4, 2026 19:38

apacheGH-48473: [CI][Python] Require numpy 2.0

475101e

apacheGH-48473: Update Pyodide to 0.27.1

7eeb4c1

Pandas 2.2.2 and numpy 2.0 upgrade followups

34131dd

Update pyodide to 0.27.1 in docker. Update pandas related comments an…

86300a6

…d docs.

apacheGH-48473: [Python] Preserve pandas 2.2 string dtype guard

4c5a080

apacheGH-48473: [Python] Pin legacy pandas test to nanoseconds

4abe534

apacheGH-48473: [Python] Update pandas type map comment

b2ceece

alippai force-pushed the alippai-numpy-20 branch from dc6ef99 to b2ceece Compare August 4, 2026 23:41

alippai requested a review from raulcd August 4, 2026 23:44

jorisvandenbossche approved these changes Aug 6, 2026

View reviewed changes

github-actions Bot added awaiting merge Awaiting merge and removed awaiting change review Awaiting change review labels Aug 6, 2026

	("3.13", "latest", "2.1.0", False),
	("3.13", "latest", "2.1.3", False),

Uh oh!

Conversation

alippai commented Aug 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

Uh oh!

github-actions Bot commented Aug 2, 2026

Uh oh!

alippai commented Aug 2, 2026

Uh oh!

kou commented Aug 3, 2026

Uh oh!

github-actions Bot commented Aug 3, 2026

Uh oh!

raulcd left a comment

Choose a reason for hiding this comment

Uh oh!

raulcd Aug 3, 2026

Choose a reason for hiding this comment

Uh oh!

jorisvandenbossche Aug 6, 2026

Choose a reason for hiding this comment

Uh oh!

raulcd commented Aug 3, 2026

Uh oh!

github-actions Bot commented Aug 3, 2026

Uh oh!

raulcd commented Aug 3, 2026

Uh oh!

github-actions Bot commented Aug 3, 2026

Uh oh!

rok left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ngoldbaum left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alippai commented Aug 3, 2026

Uh oh!

ngoldbaum left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jorisvandenbossche left a comment

Choose a reason for hiding this comment

Uh oh!

jorisvandenbossche Aug 6, 2026

Choose a reason for hiding this comment

Uh oh!

jorisvandenbossche Aug 6, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

alippai commented Aug 2, 2026 •

edited

Loading