[codex] Harden Flight pyarrow startup readiness - #8
Merged
Merged
Conversation
GTrunSec
force-pushed
the
codex/fix-flight-header-interop-timeout
branch
3 times, most recently
from
April 12, 2026 22:46
c6962a7 to
d200419
Compare
Collaborator
Author
|
Julia ecosystem is hard. |
GTrunSec
force-pushed
the
codex/fix-flight-header-interop-timeout
branch
from
April 13, 2026 03:04
6296555 to
9dd9875
Compare
GTrunSec
marked this pull request as ready for review
April 13, 2026 03:05
GTrunSec
pushed a commit
that referenced
this pull request
Sep 17, 2026
…uffers findings - #3: OwnerRegion carries @atomic closed; close!(region) marks it and runs the root's finalizers now (mmap unmaps eagerly — the Windows delete-a-mapped-file case; a foreign C-data root runs its release callback). Every raw access flows through sliceptr, whose monotonic check turns use-after-close into InvalidStateException. Idempotent; concurrent-with-close access remains the caller's contract, as with Base.close on an IO. Pinned in core tests; the facade will surface an Arrow.close!(::Table) wrapper and docs blurb. - #8: the C data entrypoints are trim-compiled and gated — the workload exports, imports, validates, releases through close!, and reaps under --trim=safe with zero verifier errors. Getting there devirtualized the adapter: formatstring_of and _validate_advisory_values_of ladders, kwcall-free _arraydata construction, Val-literal field stores, a single-assignment _newroot (boxed captures are trim-rejected), and concretely-typed release claim slots. - #10: the experimental V4 compression refusal is reworded as permanent policy (superseded by V5 BodyCompression in 2020), not prove-out scoping. - #1/#7 research recorded in docs/dev/research-flatbuffers-cdata.md: vendored-FlatBuffers assessment (keep vendoring for 3.0; extraction to a FlatBuffersGen.jl is the post-3.0 path; upstream is dormant and architecturally incompatible) and the C-data PR landscape (#561/#594/ #607-stack) with an adoption and engagement plan. The runtime's three dead Go-port carryovers (reset! with its emtpy! typo, union! on immutable tables, bytevector off-by-one) are deleted, and vtableEqual no longer allocates an IOBuffer per vtable slot on every endobject!. Gates: Pkg.test green, trim 0 errors, corpus 275/0/36, oracle 170/0/43. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Why
GitHub CI could reach the pyarrow Flight server as soon as the port opened, while the server was still not ready to answer the first real Flight RPC. That made
test/flight/header_interop.jlfail intermittently withDEADLINE_EXCEEDEDon the firstlistactionscall.Impact
Flight header interopon slower CI runnersValidation
cd /Users/guangtao/ghq/github.com/tao3k/xiuxian-artisan-workshop/.data/arrow-julia && julia -e 'using JuliaFormatter; format(".", verbose=true)'direnv exec /Users/guangtao/ghq/github.com/tao3k/xiuxian-artisan-workshop bash -lc 'cd "$PRJ_DATA_HOME/arrow-julia" && julia --project=test -e "using Test, Arrow, gRPCClient, Tables; include(\"test/flight/support.jl\"); include(\"test/flight/header_interop.jl\")"'direnv exec /Users/guangtao/ghq/github.com/tao3k/xiuxian-artisan-workshop bash -lc 'cd "$PRJ_DATA_HOME/arrow-julia" && julia --project=test -e "using Test, Arrow, gRPCClient, Tables; include(\"test/flight/support.jl\"); for _ in 1:10; include(\"test/flight/header_interop.jl\"); end"'direnv exec /Users/guangtao/ghq/github.com/tao3k/xiuxian-artisan-workshop bash -lc 'cd "$PRJ_DATA_HOME/arrow-julia" && julia --project=test -e "using Test, Arrow, gRPCClient, Tables; include(\"test/flight.jl\")"'git -C /Users/guangtao/ghq/github.com/tao3k/xiuxian-artisan-workshop/.data/arrow-julia diff --check -- test/flight/support.jl test/flight/support/grpc.jl test/flight/support/python_servers.jl