chore: [branch-54] upgrade DataFusion to 54.1.0 and bump Ballista version to 54.1.0 - #2127
Merged
andygrove merged 1 commit intoJul 22, 2026
Conversation
Move the lockfiles to the DataFusion 54.1.0 patch release, re-sync the vendored DataFusion proto files, which picks up the new `null_aware` field on `JoinNode`, and bump the Ballista crate versions to 54.1.0. The workspace requirements stay at `54` rather than pinning `54.1.0`. The python workspace has its own lockfile but depends on the ballista crates by path, so an exact requirement here forces it onto 54.1.0 as well, and datafusion-python 54.0.0 does not compile against 54.1.0 -- that release added a `schema` field to the public `RecursiveQuery` struct. Leaving the requirement at `54` lets the main workspace build against 54.1.0 while python/Cargo.lock stays on 54.0.0 until datafusion-python catches up.
andygrove
changed the title
phillipleblanc
approved these changes
Jul 22, 2026
andygrove
marked this pull request as ready for review
July 22, 2026 01:56
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
N/A
Rationale for this change
DataFusion 54.1.0 was released, so the
branch-54release branch should pick it up. This also bumps the Ballista crate versions to 54.1.0 in preparation for a 54.1.0 release.This is the
branch-54counterpart of #2126.What changes are included in this PR?
Cargo.lockandpython/Cargo.lockto the DataFusion 54.1.0 patch release.dev/update_datafusion_proto.py, which adds the newnull_awarefield toJoinNodeinballista/core/proto/datafusion.proto.dev/update_ballista_versions.py.Cargo.tomlexplaining why thedatafusion*requirements stay at54.The workspace requirements deliberately stay at
54rather than pinning54.1.0. Thepythonworkspace has its own lockfile but depends on the ballista crates by path, so an exact requirement here drags it onto 54.1.0 too, anddatafusion-python54.0.0 does not compile against 54.1.0:DataFusion 54.1.0 added a
schemafield to the publicRecursiveQuerystruct, and there is no 54.1.0 release ofdatafusion-pythonyet. Keeping the requirement at54lets the main workspace build against 54.1.0 whilepython/Cargo.lockstays on 54.0.0 untildatafusion-pythoncatches up.The lockfile also drops a duplicate
windows-sys0.53/0.60 tree that the DataFusion 54.1.0 dependency graph no longer pulls in.Are there any user-facing changes?
No API changes. The Ballista crates are versioned 54.1.0 and the scheduler, executor, CLI and benchmarks build against DataFusion 54.1.0. The Python wheel continues to build against DataFusion 54.0.0.