Skip to content

Conversation

cdce8p
Copy link
Member

@cdce8p cdce8p commented Jul 25, 2025

Python 3.14.0rc1 was released tree days ago. Add it to the build matrix.
Since there will be no ABI changes from this point forward, it's safe to use it for the wheel builder.

https://www.python.org/downloads/release/python-3140rc1/

Unfortunately the musl detection broke in 3.14 again. Added a patch to hard code it. Also added one to skip some tests which result in recursion errors.

The build uses the sigstore validation introduced in #303 since PGP signatures are no longer provided for 3.14.

@cdce8p cdce8p force-pushed the python-314 branch 2 times, most recently from 5c14336 to 1211c2d Compare July 25, 2025 13:48
@cdce8p
Copy link
Member Author

cdce8p commented Jul 31, 2025

@agners This PR would also be ready for review. The one failing test is probably just flaky and needs a restart.

@bdraco
Copy link
Member

bdraco commented Jul 31, 2025

I restarted the failing job

@agners
Copy link
Member

agners commented Jul 31, 2025

From what I can tell we did not merge Python rc versions in the past. Should we maybe use a different tag for the images and/or at least add a note in the README.md? 🤔

@cdce8p
Copy link
Member Author

cdce8p commented Jul 31, 2025

From what I can tell we did not merge Python rc versions in the past.

Yes, though one of the reasons I imagine was the low priority and the frequent CI issues. I was working on mypyc and compiling Python 3.14 for it anyway so it was fairly strait forward for me to identify the recursion errors in the test suite and add a patch to skip the relevant test cases.

Should we maybe use a different tag for the images and/or at least add a note in the README.md? 🤔

Hmm, not sure a different tag makes sense. That's just something which needs to be reverted in time and will require changes in downstream projects. How about I update the README and revert the PYTHON_LATEST change to continue pointing to 3.13 for now? Will push a new commit for it shortly.

@cdce8p
Copy link
Member Author

cdce8p commented Aug 3, 2025

The failing debian bullseye armhf build is unrelated. Looks like the package repository link doesn't work anymore. Seems it might have been removed last week / at least an earlier run did still succeed. Anyway, the next debian release is scheduled for next week so might make sense to drop support for bullseye together with it.

--
The failing link which now just returns a 404: http://deb.debian.org/debian/dists/bullseye/main/binary-armel/Packages

@agners
Copy link
Member

agners commented Aug 5, 2025

@cdce8p from my point of view we can remove bullseye already today to make this green. The images are still there just in case.

@cdce8p
Copy link
Member Author

cdce8p commented Aug 11, 2025

@agners CI looks good now. All checks passed.

Copy link
Member

@agners agners left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So far we did not release base images at rc time. I am a bit worried that folks end up using rc releases of Python accidentally. Should we maybe reflect rc in the tag? 🤔

@agners
Copy link
Member

agners commented Aug 11, 2025

Since there will be no ABI changes from this point forward, it's safe to use it for the wheel builder.

Good to know, so yeah this should make it safe to build wheels and use them post 3.14 release. That said, there could be bugs in Python 3.14 rc releases which generate "bad" Python wheels. At the very least I would still purge all Python wheels when we move to a stable 3.14 Python version.

We've discussed this PR today: Besides the above, another concern is that it will lengthen the wheels build time. Also, this time around the intention is to jump to Python 3.14 only after we drop the currently deprecated architectures (which is scheduled in December). This avoids unnecessary work put into getting the deprecated architecture working when we simply drop them a couple weeks later.

For testing it should be possible to create rc base images locally and run the build wheels locally.

So we'd rather prefer to hold off bumping at least until 3.14 is out.

@agners agners marked this pull request as draft August 11, 2025 15:06
@eshattow
Copy link
Contributor

eshattow commented Sep 2, 2025

PEP 745 – Python 3.14 Release Schedule:

  • 3.14.0 candidate 1: Tuesday, 2025-07-22
  • 3.14.0 candidate 2: Thursday, 2025-08-14
  • 3.14.0 candidate 3: Tuesday, 2025-09-16
  • 3.14.0 final: Tuesday, 2025-10-07

@cdce8p
Copy link
Member Author

cdce8p commented Sep 19, 2025

I've updated the PR after the release of Python 3.14.0rc3 this week. It includes the backport of python/cpython#137864 which fixes the musl version detection, so the custom patch for that isn't necessary here anymore.

--

Since there will be no ABI changes from this point forward, it's safe to use it for the wheel builder.

Good to know, so yeah this should make it safe to build wheels and use them post 3.14 release. That said, there could be bugs in Python 3.14 rc releases which generate "bad" Python wheels. At the very least I would still purge all Python wheels when we move to a stable 3.14 Python version.

Deleting the wheels isn't necessary. Python itself is dynamically linked and the ABI is now frozen. From the release post

We strongly encourage maintainers of third-party Python projects to prepare their projects for 3.14 during this phase, and publish Python 3.14 wheels on PyPI to be ready for the final release of 3.14.0, and to help other projects do their own testing. Any binary wheels built against Python 3.14.0 release candidates will work with future versions of Python 3.14.

https://discuss.python.org/t/python-3-14-0rc3-is-go/103815

--

We've discussed this PR today: Besides the above, another concern is that it will lengthen the wheels build time. Also, this time around the intention is to jump to Python 3.14 only after we drop the currently deprecated architectures (which is scheduled in December). This avoids unnecessary work put into getting the deprecated architecture working when we simply drop them a couple weeks later.

I also thought we might want to do it that way. Though I would prefer not to wait until December before releasing support for 3.14. Using matrix.exclude here works well enough IMO. Regarding the wheel build times, I don't think these are an issue. First of this PR is just the preparation to get all the pieces in place. There are still 2-3 blockers for Home Assistant core, I'm tracking currently. Second, most dependencies already publish wheels for musllinux arm64 and aarch64 so this isn't a big issue.

For testing it should be possible to create rc base images locally and run the build wheels locally.

Yes, it is. My idea was just to put all the pieces in place early on, to be ready. When 3.14.0 is released in Oktober.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants