Repository navigation
Add experimental Project AST JIT for integral add and multiply [databricks] - #15312
Merged
Merged
Conversation
Collaborator
Author
|
@greptile full review |
Signed-off-by: Haoyang Li <haoyangl@nvidia.com>
Contributor
 [Medium impact] No blocking issue was found in the reviewed changes. SummaryAdds experimental Project AST JIT for non-ANSI integer and long addition and multiplication.
Diagram%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Project expressions] --> B[Check enabled backends]
B --> C[Build dependency tiers]
C --> D[Regular GPU expressions]
C --> E[Interpreted AST expressions]
C --> F[Grouped JIT expressions]
F --> G[Build or reuse task-local program]
D --> H[Ordered output batch]
E --> H
G --> H
G --> I[Close at task completion]
Reviews (12) · Last reviewed commit: "Merge branch 'main' into project-ast-jit..." · Reviewed by Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Signed-off-by: Haoyang Li <haoyangl@nvidia.com>
Collaborator
Author
|
Java binding is in 26.10, waiting for main branch to switch... |
3 of 8 tasks
Signed-off-by: Haoyang Li <haoyangl@nvidia.com>
Signed-off-by: Haoyang Li <haoyangl@nvidia.com>
Collaborator
|
NOTE: release/26.08 has been created from main. Please retarget your PR to release/26.08 if it should be included in the release. |
Signed-off-by: Haoyang Li <haoyangl@nvidia.com>
Signed-off-by: Haoyang Li <haoyangl@nvidia.com>
Signed-off-by: Haoyang Li <haoyangl@nvidia.com>
Signed-off-by: Haoyang Li <haoyangl@nvidia.com>
thirtiseven
added a commit
that referenced
this pull request
Aug 4, 2026
…ut] (#15377) Related to #8954. ### Description The motivation for this PR is to prepare for AST JIT work. Legacy AST Project execution is currently all-or-nothing: if any top-level Project output cannot be represented by the legacy AST backend, eligible sibling outputs cannot use AST either. This change selects the legacy AST backend independently for each top-level Project expression. Supported fixed-width outputs use AST while unsupported outputs in the same Project continue through the regular GPU expression path. Top-level null literals also remain on the regular projection path so their outputs can reuse its cached null vector. It uses the existing `spark.rapids.sql.projectAstEnabled` configuration and does not change result semantics or add a user-facing configuration. This is independent of the experimental AST JIT work in #15312. It refactors the existing legacy AST path and does not depend on AST JIT, LTO, or precompiled fragments. This also changes physical-plan rendering: AST projections now appear under GpuProject as `AST(...) AS x` instead of using a dedicated `GpuProjectAstExec` node. This does not change result semantics, APIs, or configuration. Implementation details: - Represent each eligible output with a lightweight `GpuProjectAstExpression`. - Keep top-level null literals on the regular projection path so multiple null outputs can reuse its cached null vector; non-null literals remain AST-eligible. - Share one cuDF input `Table` across all AST outputs evaluated for the same tier and batch. - Preserve TieredProject common-subexpression elimination for duplicate outputs and shared subtrees. - Route fused higher-order-function projections through the same expression evaluator so AST outputs retain the shared input Table. - Close compiled legacy AST expressions at task completion, including retry execution paths. Performance testing: - Benchmark script: [project_ast_per_expression_perf.scala](https://gist.github.com/thirtiseven/f20355b1e8c32444fb11a39fa3717545) - Environment: Spark 3.5.2 with `local[4]`, plugin `26.08.0-SNAPSHOT`, and 2 NVIDIA RTX 5880 Ada Generation GPUs. - Workload: 20 million Parquet rows in 8 partitions, 84 projected outputs, one warmup, and five measured iterations. A global GPU aggregate materializes every projected output. - `LIBCUDF_JIT_ENABLED=0` isolates the legacy AST backend. AST off/on order and case order are reversed on alternating iterations. - Speedup is AST off / AST on; values above 1 mean AST is faster. The Project metric is `opTimeLegacy`, accumulated across tasks, so it is not directly comparable to E2E wall-clock time. | Case | Expressions | Coverage / CSE shape | AST tiers (off/on) | AST outputs (off/on) | AST off/on E2E median (ms) | E2E speedup | AST off/on Project op median (ms) | Project op speedup | |---|---|---|---|---:|---:|---:|---:|---:| | `broad_unique_ast` | 84 unique AST-compatible | 42 AST operator families | `[0]` / `[84]` | 0 / 84 | 670.249 / 589.848 | 1.136x | 684.796 / 356.251 | 1.922x | | `whole_output_duplicates` | 42 AST expressions, each projected twice | whole-output CSE | `[0,0]` / `[42,0]` | 0 / 84 | 601.863 / 526.478 | 1.143x | 355.906 / 155.668 | 2.286x | | `cheap_partial_cse` | 84 AST-compatible | one shared add | `[0,0]` / `[1,84]` | 0 / 84 | 516.149 / 467.530 | 1.104x | 214.336 / 189.046 | 1.134x | | `expensive_partial_cse` | 84 AST-compatible | one shared transcendental subtree | `[0,0]` / `[1,84]` | 0 / 84 | 476.142 / 440.932 | 1.080x | 239.745 / 175.069 | 1.369x | | `mixed_half` | 42 AST-compatible + 42 regular GPU | mixed execution | `[0]` / `[42]` | 0 / 42 | 793.670 / 712.257 | 1.114x | 1167.327 / 868.695 | 1.344x | No median regression was observed in the five main benchmark cases in either E2E or Project op time. #### Top-level literal routing follow-up A targeted follow-up compared top-level literals on the regular and AST paths. It used 20 million rows, 84 Project outputs, two warmups, and 15 alternating measured iterations. The benchmark directly executed and consumed the columnar `GpuProject`, avoiding Catalyst constant folding and unrelated aggregate, shuffle, or ColumnarToRow work. Speedup is regular / AST; values above 1 mean AST is faster. | Case | Expressions | Regular/AST E2E median (ms) | E2E speedup | Regular/AST Project op median (ms) | Project op speedup | |---|---|---:|---:|---:|---:| | `literal_only` | 84 unique non-null long literals | 61.418 / 60.952 | 1.008x | 117.137 / 104.202 | 1.124x | | `mixed_ast_literals` | 42 AST-compatible expressions + 42 unique non-null long literals | 231.422 / 230.816 | 1.003x | 241.066 / 239.654 | 1.006x | | `null_literal_only` | 84 duplicate double null literals | 33.257 / 137.528 | 0.242x | 8.224 / 407.906 | 0.020x | Non-null literals were neutral in E2E time in both the literal-only and mixed workloads, so there is no evidence for excluding all top-level literals from AST. Null literals were different: the regular projection was 4.1x faster E2E and 49.6x faster in Project op time because it can reuse its cached null vector across outputs, whereas per-expression AST evaluates the null outputs independently. Based on these results, this PR keeps only top-level null literals on the regular projection path and leaves non-null literals AST-eligible. ### TPC-H/TPC-DS coverage We also performed physical-plan sweeps over stock TPC-H/NDS-H and TPC-DS workloads. - TPC-H/NDS-H with the standard Decimal schema produced no legacy Project AST expressions across 24 physical plans. - TPC-DS produced only three AST expressions across 102 physical plans. Two operate on a small filtered date dimension, and the remaining expression is in a post-aggregation Project. These workloads therefore spend nearly all of their time in unrelated scans, joins, and aggregations and do not provide a meaningful performance signal for this change. The performance results above use purpose-built AST-only and mixed-expression Projects so that the affected execution path is exercised directly, while still materializing every projected expression. ### Checklists Documentation - [ ] Updated for new or modified user-facing features or behaviors - [x] No user-facing change Testing - [x] Added or modified tests to cover new code paths - [ ] Covered by existing tests (Please provide the names of the existing tests in the PR description.) - [ ] Not required Performance - [x] Tests ran and results are added in the PR description - [ ] Issue filed with a link in the PR description - [ ] Not required --------- Signed-off-by: Haoyang Li <haoyangl@nvidia.com> Co-authored-by: Igor Peshansky <7594381+igorpeshansky@users.noreply.github.com>
Signed-off-by: Haoyang Li <haoyangl@nvidia.com>
thirtiseven
changed the title
Signed-off-by: Haoyang Li <haoyangl@nvidia.com>
Collaborator
Author
|
@greptile full review |
Collaborator
Author
|
I filed #15853 as a follow-up to revisit the multi-output AST JIT group limits once the feature is more mature and supports more operators. For now, this PR uses conservative defaults of 384 unique operations and 32 outputs per group as safety guards. |
Signed-off-by: Haoyang Li <haoyangl@nvidia.com>
Signed-off-by: Haoyang Li <haoyangl@nvidia.com>
Collaborator
|
This is still in "draft" — @thirtiseven, is this ready for review? |
Collaborator
Author
No, I'm still working on it. I updated the solution and it needs more tests. It also depends on NVIDIA/cudf#23828 |
rapids-bot Bot
pushed a commit
to NVIDIA/cudf
that referenced
this pull request
Sep 15, 2026
…on (#23828) This PR exposes the libcudf AST JIT capabilities introduced by #23615 and #23621 through the Java API, and adds a reusable Java binding for `cudf::transform_program` introduced by #23648. The execution backend is selected when an expression is compiled: - `AstExpression.compile()` preserves the existing behavior and produces an expression compatible with default AST consumers. - `AstExpression.compileJit()` produces a JIT-specific expression. Each `CompiledExpression` owns only the native AST tree and literal representation required by its compilation mode. - `CompiledExpression.computeColumn(Table)` executes using the compiled expression's mode, avoiding a separate `computeColumnJit` API and avoiding duplicate native AST trees. - `CompiledExpression.computeTableJit(Table, CompiledExpression...)` evaluates multiple JIT-compiled expressions in one `compute_table_jit` call. Output order follows the supplied expression order, and libcudf Row IR can eliminate structurally equivalent subexpressions across outputs. For JIT-compiled expressions, literals are converted to one-row columns once during compilation and retained for repeated evaluation. Temporary scalar owners are released after construction completes. Default-compiled expressions retain the existing scalar-backed representation and do not incur the JIT literal-column allocation cost. The new `AstJitProgram` API exposes reusable AST JIT execution: - `AstJitProgram.compile(Table, CompiledExpression...)` lowers one or more JIT expressions and retrieves a `cudf::transform_program` specialized to the referenced input-column types and physical nullability. - `AstJitProgram.computeTable(Table)` reuses the lowered program and retrieved kernel across compatible input tables. Row counts and unreferenced columns may differ between evaluations. - The program does not retain the schema table or source expressions and remains valid after they are closed. To support this ownership contract, `cudf::transform_program` now copies `scalar_column_view` literal inputs into program-owned columns. Both default- and JIT-compiled expressions can retain device-backed literal values, so compiled expressions in both modes, along with reusable programs, are registered with the RMM cleanup path. Reusable programs amortize repeated AST lowering, kernel lookup, and warm dispatch-side overhead. They do not reduce first-use or cold JIT compilation latency. The binding also: - rejects compilation-mode mismatches, including passing default-compiled expressions to JIT APIs; - prevents JIT-compiled expressions from being passed to default-only consumers such as join and scan predicates; - validates null, closed, and schema-incompatible inputs; - preserves the lifetime of input tables and compiled expressions across JNI calls. The `compileJit` and direct multi-output APIs are consumed by NVIDIA/cudf-spark#15312 to execute compatible Project expressions as multi-output AST JIT waves. Authors: - Haoyang Li (https://github.com/thirtiseven) - MithunR (https://github.com/mythrocks) Approvers: - Igor Peshansky (https://github.com/igorpeshansky) URL: #23828
Signed-off-by: Haoyang Li <haoyangl@nvidia.com>
Signed-off-by: Haoyang Li <haoyangl@nvidia.com>
Collaborator
|
NOTE: release/26.10 has been created from main. Please retarget your PR to release/26.10 if it should be included in the release. |
Signed-off-by: Haoyang Li <haoyangl@nvidia.com>
Signed-off-by: Haoyang Li <haoyangl@nvidia.com>
Signed-off-by: Haoyang Li <haoyangl@nvidia.com>
thirtiseven
marked this pull request as ready for review
October 16, 2026 09:40
Collaborator
Author
|
This PR is ready for review again. It now supports running eligible subtrees with JIT, using cuDF JIT’s CSE, and mixing JIT with legacy AST in the same Project. I opened #16385 to track the broader AST-JIT work, with #16383 covering cold-start costs and #16384 covering cost-based backend selection. If this PR feels too large to review, I’m happy to split it up. |
revans2
requested changes
Oct 21, 2026
revans2
left a comment
revans2
left a comment
Collaborator
There was a problem hiding this comment.
It is looking a lot better, but my AI and I did find two blocking issues.
Signed-off-by: Haoyang Li <haoyangl@nvidia.com>
…tion - Judge interpreted simple arithmetic on the region left after materialization - Do not materialize a value only for sharing when it would reduce an interpreted consumer to simple arithmetic - Keep planning-time wrappers only when the output's final root runs on that backend - Enable Project AST backends only through explicit binder flags - Remove Retryable from GpuAstJitExpression - Report unselected JIT candidates only when JIT is enabled Signed-off-by: Haoyang Li <haoyangl@nvidia.com>
thirtiseven
changed the title
Collaborator
Author
|
build |
revans2
approved these changes
Oct 27, 2026
Collaborator
Author
|
I'm going to merge this PR. If there are any new comments, I'm happy to address them in follow-ups. I also reran the cold-start benchmark on the current PR head with the 2026-10-26 cuDF nightly. JIT compilation is now 2–3x faster than in September. The gain comes from cuDF's JIT improvements, not from changes in this PR.
PR description updated. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Contributes to #10640
Description
This PR adds experimental cuDF AST JIT execution to
GpuProjectExecand unifies its planning with interpreted (legacy) AST and regular GPU projection. It builds on #15377 and groups compatible JIT outputs for cuDF Row IR CSE, while interpreted AST uses plugin-side CSE and the existing tiered projection lifecycle.JIT operator coverage remains limited to non-ANSI
IntegerTypeandLongTypeaddition and multiplication. The internalspark.rapids.sql.projectAstEnabledandspark.rapids.sql.projectAstJitEnabledswitches independently enable interpreted AST and JIT; both remain disabled by default. This PR does not solve JIT cold-start cost or enable either backend by default.Planning and execution
The shared Project planner uses existing per-node operator/type eligibility to discover supported subtrees and build dependency tiers. With tiering enabled, eligible children can be extracted below unsupported roots and from the GPU inputs of CPU bridges without depending on CSE to expose them first. Only deterministic values without side effects move to an earlier tier, where they are computed for every row, so an operand that
AND,OR,IF, orCASE WHENwould skip is never evaluated early if it can fail. Lambda bodies are not split. With tiering disabled, only complete output expressions are selected.The placement policy is:
This is a fixed placement policy, not a profitability cost model. Sharing can introduce materialization overhead, and enabling JIT does not establish that compilation will be amortized. Compilation-, execution-, and materialization-aware optimization remains follow-up work (#16384).
Each JIT root is compiled explicitly with
AstExpression.compileJit. Compatible roots useAstJitProgram.compileand the reusable program'scomputeTablefor multi-output execution. Single-output execution also reuses the task-local program. The internalspark.rapids.sql.projectAstJitMultiOutputEnabledswitch defaults to true and only applies when JIT is enabled.Project and extracted BHJ/BNLJ build-side projections pass both backend switches explicitly to the tiered binder. Generic binders enable neither Project AST backend and drop existing AST wrappers unless the caller enables the matching backend. The build-side projection that the BHJ hash-build cache retains for rebuilds in later tasks therefore uses regular GPU evaluation; #16509 tracks AST and AST JIT support there.
spark.rapids.sql.explain=ALLreports final bound tiers asAST JIT,AST Interpreted, or regular GPU, including selections exposed during tier planning. The Spark plan string shows a backend for a Project output only when that output's final root runs on it; subexpressions extracted to earlier tiers appear only in the final-selection log.Retry and resource lifecycle
Compiled expressions and JIT programs are created lazily inside the projection retry boundary, including when Project split retry is disabled, reused across retries and batches within a task, and closed at task completion. Multi-output results retain ARM ownership, failure cleanup, output ordering, and null propagation. Extracted BNLJ projections use the same spillable, OOM-retry-aware helper as BHJ.
Implementation and validation were assisted by OpenAI Codex.
cuDF backend status and remaining validation
rapidsai/cudf#23828, which provides the Java/JNI AST JIT bindings, is merged. Local correctness validation below used a locally installed 26.12 snapshot containing these APIs; it does not establish the contents or success of a published Jenkins nightly artifact.
The current backend generates CUDA source from Row IR. Cold compilation and precompiled operator-fragment/LTO integration remain separate work (#16383). The 2026-10-27 PCH benchmark below was rerun on the current PR head; the other performance sections retain their original artifact versions and cache-state descriptions and are dated evidence rather than reruns of the latest implementation.
Testing
Local validation on 2026-10-22 used Spark 3.5.2 (
buildver=352), Java 17, and one RTX 5880 Ada GPU, with the 2026-10-21 26.12 snapshot of cudf-spark-jnidc2af3a4e358bb8f1c3e32e46502307b49e39792and cuDFb83f75be57ddf3850b0178c805dd4d6d9178d339:mvn clean install -Dbuildver=352 -DskipTests: passed, including test-source compilation and all-module Scalastyle with no errors or warnings.GpuProjectAstPlannerSuite,GpuProjectAstJitSuite,ProjectExprSuite,ProjectSplitRetrySuite,GpuBroadcastNestedLoopJoinRetrySuite,GpuBroadcastHashJoinPostProjectionSuite, andGpuArrayHofFusionSuite: 125 passed, no failures or aborted suites.ast_test.pyplustest_cpu_bridge_with_ast_backendsandtest_cpu_bridge_short_circuit_with_ast_backends: 247 passed, 1 existing xfail. The xfail is the unchangedtest_columnar_powexpectation.Coverage includes independent backend switches, plugin/cuDF CSE placement, maximal cross-backend sharing and its exception for interpreted consumers, simple arithmetic before and after materialization and CSE, compound AST below unsupported roots, CPU bridge regions, skipped operands with side effects under
AND/OR/IF/CASE WHEN, planning-time wrappers that match the final selection, tiering and multi-output on/off, INT/LONG CPU/GPU equality, serialization/type eligibility, retry, resource cleanup, and the cached BHJ build-side projection.Other Spark shims, Scala 2.13, and Databricks were not run locally.
Performance
Verified PCH benchmark rerun on the PR head: 20M and 200M rows, 1–16 outputs (2026-10-27)
This reruns the 2026-09-15 matrix below on the current PR head, including the latest planner changes, with the same machine, inputs, and protocol. Its cuDF includes later JIT compile-time improvements, so these numbers replace the September compile and first-use costs.
613cb0635a1ac9bccf5202d60b8c5557f43f5fe5(PR head), built withmvn clean install -Dbuildver=352 -DskipTests; JNI9bcf05eac4990416cdb5f2741187ad128ebfe811(2026-10-26 nightly); cuDF01a7871610d07efd0dbd5828116e9e15eea701b6.local[4], one RTX 5880 Ada GPU (CUDA_VISIBLE_DEVICES=0), Intel Core i7-9700K (8 cores; each NVRTC compilation runs on one core). All arms use NVTX-only Nsight Systems profiling, no CPU sampling, fresh processes and isolated kernel caches; runs are serial. Inputs and expression shapes are unchanged.enableAstflag, and two metrics that only existed in the LTO prototype. Measurement and verification logic are unchanged.Compiler and first-action times
Values are medians across three processes. Compiler time is target-window
rtcx:compilehost duration; action time is wall-clockcollect()after planning. Spark startup, non-JIT GPU bootstrap, the optional distinct one-row PCH probe, and runtime-version verification are excluded.multi_independentmulti_independentmulti_independentmulti_independentmulti_independentmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addmulti_independentmulti_independentmulti_independentmulti_independentmulti_independentmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addProject op-time and end-to-end speedups
All ratios are Regular / JIT; above 1 means JIT is faster.
Cold-PCH OP= regular first-action Project op time / cold-PCH JIT first-action Project op time.Warm-PCH OP= the same regular first-action Project op time / warm-PCH JIT first-action Project op time.Hot OPcompares both modes' hot actions. Each process contributes the median of its five hot actions; the reported value is the ratio of the medians across three processes. Hot JIT values use the warm-PCH arm.multi_independentmulti_independentmulti_independentmulti_independentmulti_independentmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addmulti_independentmulti_independentmulti_independentmulti_independentmulti_independentmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addOPis the target Project'sopTimeLegacy: accumulated host time across tasks, including construction, evaluation, and waiting. It excludes the surrounding read/aggregate work as timed by their own operators, but is not isolated GPU kernel time. Task-summed OP cannot be subtracted from query wall time or used as an exact decomposition of scan and aggregate latency. The much smaller cold/warm OP ratios reflect compilation/waiting inside Project, compared with regular Project work alone.Interpretation
Project op times, process ranges, and changes from 2026-09-15
All OP values are task-summed milliseconds. Hot values are medians of per-process hot medians.
multi_independentmulti_independentmulti_independentmulti_independentmulti_independentmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addmulti_independentmulti_independentmulti_independentmulti_independentmulti_independentmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addCompile ranges are across three processes. Hot ranges describe the three per-process medians, not individual actions.
multi_independentmulti_independentmulti_independentmulti_independentmulti_independentmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addmulti_independentmulti_independentmulti_independentmulti_independentmulti_independentmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addMedians from the 2026-09-15 run → this rerun.
multi_independentmulti_independentmulti_independentmulti_independentmulti_independentmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addmulti_independentmulti_independentmulti_independentmulti_independentmulti_independentmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addThe measurements below predate the latest planner changes (side-effect-aware tier boundaries, the sharing exception for interpreted consumers, and the interpreted region rule) and were not rerun for them.
Interpreted AST placement: idle-host A/B rerun (2026-10-10)
This comparison isolates the rule that leaves standalone simple arithmetic on regular GPU, on top of the unified planner. The implemented planner matches the tested variant. JIT was disabled throughout; these results do not measure JIT speedup or cold start.
The benchmark used 20M Parquet rows, 16 outputs, eight scan partitions/batches, Spark 3.5.2, Java 17,
local[4], one RTX 5880 Ada, one concurrent GPU task, a 256 MiB batch target, and a verified 2 GiB pinned pool. Three paired runs used six fresh JVMs in A/B, B/A, A/B order, with two warmups and five timed actions per case/mode: 240 timed actions and 96 warmups. CPU/GPU output aggregates, 49,152 small-input row comparisons, and plan assertions passed. Monitoring found no concurrent native compiler processes.Each absolute time is the median of three per-JVM medians. Reductions are calculated within matched JVM pairs; ranges are the three paired reductions, not confidence intervals.
The cheap case retained CSE but removed all 17 standalone AST fragments. The expensive case retained one fused shared AST instead of 17 fragments. The mixed case reduced 32 fragments in two tiers to seven compound AST expressions in one tier. The broad control retained 16 AST expressions in one tier.
Projectis task-summedopTimeLegacyhost time, including preparation and waiting; it is not GPU kernel time and cannot be added to query wall time. Cheap/expensive query reductions span zero, so these measurements do not establish consistent end-to-end speedups. The updated expensive legacy query remained slightly slower than regular GPU despite its improved Project metric. These four warm workloads support the narrow placement rule, not a general claim that legacy AST should be enabled by default.This performance campaign used frozen cudf-spark-jni
16fcee1cd31ad8c07ec22eb8f0eae205f1ab4aa6and cuDFc4bb5ea7f70db0ca5201ff534358b7f2f562cc15artifacts. The newer native artifacts in the Testing section were used for correctness/build validation, not to regenerate these timings. The full historical performance sections follow unchanged.2026-09-15 PCH benchmark on the September cuDF build (superseded by the rerun above)
Verified PCH benchmark: 20M and 200M rows, 1–16 outputs (2026-09-15)
The completed matrix covers both
multi_independentandmulti_shared_add, five output widths (1, 2, 4, 8, 16), and regular GPU / cold-PCH JIT / warm-PCH JIT at both input sizes. 180 fresh Spark processes and 1,080 target actions passed validation. Each cell uses three processes; each process runs one first action and five hot actions. The 20M matrix reuses the verified regular/warm-PCH results and adds the missing 15 independent cold-PCH processes. The 200M matrix is a new complete run.4f907f0b3d0494276ba069325901a1fe5a40ed97, JNI101de1ba68291f10231c8cdcdbadd6cde6e282c9, cuDF7edcf282f5e4cea38bb2c18c385e27cdf5a6599b.local[4], one RTX 5880 Ada GPU (CUDA_VISIBLE_DEVICES=0). All arms use NVTX-only Nsight Systems profiling, no CPU sampling, fresh processes and isolated kernel caches; runs are serial.(a + k) * (b + m); shared roots are(a + b) * (c + k), with different literals per output. A GPU aggregate consumes every output.mvn clean install -Dbuildver=352 -DskipTests; JNI was already built. The benchmark-only JAR adds old-name aliases for the renamed private version metadata, with unchanged executable payloads.Compiler and first-action times
Values are medians across three processes. Compiler time is target-window
rtcx:compilehost duration; action time is wall-clockcollect()after planning. Spark startup, non-JIT GPU bootstrap, the optional distinct one-row PCH probe, and runtime-version verification are excluded.multi_independentmulti_independentmulti_independentmulti_independentmulti_independentmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addmulti_independentmulti_independentmulti_independentmulti_independentmulti_independentmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addProject op-time and end-to-end speedups
All ratios are Regular / JIT; above 1 means JIT is faster.
Cold-PCH OP= regular first-action Project op time / cold-PCH JIT first-action Project op time.Warm-PCH OP= the same regular first-action Project op time / warm-PCH JIT first-action Project op time.Hot OPcompares both modes' hot actions. Each process contributes the median of its five hot actions; the reported value is the ratio of the medians across three processes. Hot JIT values use the warm-PCH arm.multi_independentmulti_independentmulti_independentmulti_independentmulti_independentmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addmulti_independentmulti_independentmulti_independentmulti_independentmulti_independentmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addOPis the target Project'sopTimeLegacy: accumulated host time across tasks, including construction, evaluation, and waiting. It excludes the surrounding read/aggregate work as timed by their own operators, but is not isolated GPU kernel time. Task-summed OP cannot be subtracted from query wall time or used as an exact decomposition of scan and aggregate latency. The much smaller cold/warm OP ratios reflect compilation/waiting inside Project, compared with regular Project work alone.Interpretation
astJitProgramBuildTimeandastJitEvalTimeare task-summed host times. In the 200M, 16-output independent warm-PCH case, first-action program-build time is 10.36 seconds accumulated across tasks, while the trace records one 2.58-second compilation. Waiting explains why these metrics cannot be treated as serial query latency.Within each input size, all output-sum checksums matched across arms, rounds, and actions. GPU scan/Project/aggregate and tier/root assertions passed; JIT metrics confirmed the complete row count, and regular runs reported zero for all six JIT metrics. The existing PCH verifier's 10 tests passed during harness validation. Build/runtime compatibility details and earlier measurements are retained below.
Current-run absolute Project op times and dispersion
All OP values below are task-summed milliseconds. Hot values are medians of per-process hot medians.
multi_independentmulti_independentmulti_independentmulti_independentmulti_independentmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addmulti_independentmulti_independentmulti_independentmulti_independentmulti_independentmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addCompile ranges are across three processes. Hot ranges describe the three per-process medians, not individual actions.
multi_independentmulti_independentmulti_independentmulti_independentmulti_independentmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addmulti_independentmulti_independentmulti_independentmulti_independentmulti_independentmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addFirst-action ranges are included to expose startup variation that remains after the excluded bootstrap and planning steps.
multi_independentmulti_independentmulti_independentmulti_independentmulti_independentmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addmulti_independentmulti_independentmulti_independentmulti_independentmulti_independentmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addInitial verified 20M sweep before independent cold-PCH completion
The regular and warm-PCH cells below are reused in the completed matrix above; they are not additional independent samples. This initial report is retained for provenance.
Verified PCH benchmark: 1–16 outputs (2026-09-15)
Reran the realistic-width sweep at cudf-spark
4f907f0b3d0494276ba069325901a1fe5a40ed97, using JNI101de1ba68291f10231c8cdcdbadd6cde6e282c9and cuDF7edcf282f5e4cea38bb2c18c385e27cdf5a6599b. The plugin was rebuilt withmvn clean install -Dbuildver=352 -DskipTests(all 14 modules successful); JNI was already built. The benchmark JAR adds old-name aliases for the renamed private version-metadata resource so this branch can start with the rebuilt dependency; executable payloads are unchanged.75 fresh Spark processes / 450 target actions passed plan, checksum, and metric checks. Each process executes one first action and five hot actions. First-action and compiler times are medians across three processes; hot results use the median of the three per-process hot medians.
local[4], one RTX 5880 Ada GPU (CUDA_VISIBLE_DEVICES=0).multi_independent:(a + k) * (b + m), with different literals per output.multi_shared_add:(a + b) * (c + k), sharinga + bacross outputs.multi_independentmulti_independentmulti_independentmulti_independentmulti_independentmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addHot Regular/JIT > 1means JIT is faster. A dash means the cold-PCH arm was not measured for that workload. Compile time is target-windowrtcx:compilehost duration. First/hot action time is wall-clockcollect()after planning. Spark startup, the non-JIT GPU bootstrap, the optional distinct one-row PCH probe, and the runtime-version probe are excluded. These are target-action measurements, not total executor-startup latency.Cold-run interpretation and new metrics
a + bonce; JIT fusion primarily avoids intermediate-column traffic in that case.astJitProgramBuildAttempts, zeroastJitProgramCacheHits, fourastJitEvalAttempts, and 20 millionastJitEvalRows. Each task processes one batch, leaving no within-task program-reuse opportunity. Repeated actions construct fresh task-local programs while reusing the native kernel cache.astJitProgramBuildTimeincludes native cache lookup, compilation, and waiting and is summed across tasks. For 16 independent outputs its first-action median is 10.56 seconds across four tasks, while the trace records one 2.64-second compilation.astJitEvalTimeis also task-summed host time and excludes explicit program construction; it is not GPU kernel time. Neither metric should be added to query wall time or the enclosing Project metric.All output-sum checksums matched across arms, rounds, and actions. GPU scan/Project/aggregate and tier/root assertions passed, and all six JIT metrics were zero in regular runs. The existing PCH verifier's 10 tests also passed.
The measured backend remains CUDA source JIT generated from Row IR. Connecting it to precompiled operator fragments is separate work; this benchmark does not measure LTO or assume a 30-ms compile/link time.
Current-run dispersion and cold-PCH first-action timings
multi_independentmulti_independentmulti_independentmulti_independentmulti_independentmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addHot ranges describe the three per-process medians, not individual actions. Small hot differences should be interpreted with these ranges.
Superseded September 11 width sweep and single-sample PCH validation
These earlier results are retained for comparison. The verified three-process sweep above supersedes their cache-state assumptions and one-hot-action estimates.
Realistic output widths: 1–16 (2026-09-11)
A width sweep measures 1, 2, 4, 8, and 16 output expressions over 20 million Parquet rows.
multi_independentevaluates(a + k) * (b + m)with different literals per output;multi_shared_addevaluates(a + b) * (c + k), sharinga + bacross outputs. Each tested group has one logical JIT program and one actual compiler invocation, including the 16-output cases.Each value is the median of three fresh Spark processes on one RTX 5880 Ada GPU (
CUDA_VISIBLE_DEVICES=0), Spark 3.5.2, Java 17,local[4], and four realized scan partitions. A GPU aggregate consumes every output. Plan assertions check GPU execution, tiers, JIT roots, and shared-subtree placement; a separate 16-output check passed regular/JIT checksum comparison for both workloads.These measurements used cudf-spark
a0a891c7e99d9e14b71e615c34d3a8c51b12d7e8, JNI101de1ba68291f10231c8cdcdbadd6cde6e282c9, and cuDF3e810df236a0af2f082d5066f46dc11eb3335cb5. They are dated benchmark evidence, not a rerun of the latest PR head.“Prewarmed” means a distinct one-row JIT probe ran before the target, while the target kernel remained cold. The September 11 logs record that probe but do not record per-target PCH reuse or the loaded NVRTC version, so this table does not label that state as verified PCH-warm. “No probe” omits the JIT probe. Both use fresh processes and isolated kernel caches.
multi_independentmulti_independentmulti_independentmulti_independentmulti_independentmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addmulti_shared_addCompile times are target-window
rtcx:compilehost durations; first-action times are wall-clockcollect()durations after planning. Spark startup, non-JIT GPU bootstrap, and the optional JIT probe are excluded. “Hot” immediately repeats the target in the same process;Regular/JIT > 1means JIT is faster. There is only one hot action per process, so small hot differences are directional rather than a robust throughput estimate. “—” means that arm was not measured. JIT samples were collected with NVTX-only Nsight Systems tracing; the regular baseline was not profiled.Cold-run interpretation
a + bonce; keeping it inside the JIT group saves intermediate-column traffic. The observed hot ratios span 0.928x–1.137x.A separate single-sample validation on 2026-09-15 verified NVRTC 12.9 and per-target PCH reuse for the one-output shared case: compiler time was 490.7 ms with verified warm PCH, versus 2177.6 ms with verified PCH creation in the no-probe control. This supports the remaining first-use-cost observation, but is not a replacement for the three-sample width sweep or verification of its historical PCH state.
The current AST-JIT path still generates and compiles CUDA source from Row IR. Connecting it to precompiled operator fragments remains separate work. No current-path LTO timing or 30-ms-per-program assumption is included in these results.
Earlier benchmarks: warm throughput, 60-output stress cases, cache controls, and historical LTO POC
The following results were collected earlier with the workloads and artifacts described in each subsection. They are retained as historical evidence; the 60-output cases are wider stress controls and should not be generalized to the 1–16-output sweep above.
Warm throughput
The following is a same-process warm-throughput comparison, not a fresh-process cold-run measurement:
local[4];The modes are regular GPU Project, per-expression AST JIT, and multi-output AST JIT.
Regular/MultiandPer-expression/Multivalues above 1 mean multi-output JIT is faster.Project opis the cumulativeopTimeLegacySQL metric across parallel tasks; it is supporting evidence and is not directly comparable to wall-clock time.single_chainmulti_independentmulti_shared_addmixed_wavessingle_chainhas only one JIT root, so it does not activate multi-output execution and serves as a noise control. Grouping 60 independent roots is approximately neutral versus per-expression JIT at the end-to-end level in this run.When all 60 outputs share
a + b, keeping that subtree visible to cuDF for cross-output CSE and consolidating execution improves E2E by 19.7% over per-expression JIT and 22.5% over regular GPU Project. The cumulative Project-op per-expression/multi ratio is 3.018x. For the three-wave mixed workload, multi-output improves E2E by 10.3% over per-expression JIT and 14.2% over regular GPU Project.Because all modes share one Spark process after warmup, process-local compiler and kernel caches may be shared. These results support the steady-state wave/grouping policy but do not establish a cold-start improvement or a general cost policy for operators beyond the add/multiply scope.
Cold start and amortization
A separate benchmark measures first-use latency in fresh Spark processes after an excluded non-JIT GPU bootstrap. The 20-million-row and 200-million-row runs use the same expression shapes and 60-output workloads. Normal Spark file-split planning was retained, with four and 18 realized scan partitions respectively. Each value is the median of three fresh processes, and cross-mode checksums passed.
Cold Regular/JITandPCH Regular/JITare speedups: values above 1 mean JIT is faster.single_chainmulti_shared_addmixed_wavessingle_chainmulti_shared_addmixed_wavesAt 20 million rows, legacy AST first-use latency was 342.261 ms, 865.344 ms, and 913.246 ms for the three cases, close to regular Project. Empty-cache AST-JIT is 7.1–8.3x slower than regular Project at 20 million rows and remains 2.4–2.7x slower after increasing the input by 10x. PCH removes 25–80% of the cold-to-hot gap depending on expression shape, but does not make first-use JIT faster than regular Project in these workloads.
An earlier NVRTC 12.8 trace separately measures compiler work.
Output expressionscounts JIT roots. Both arms enable automatic PCH:PCH coldcreates it during the first target compilation, whilePCH warmfirst compiles a distinct one-row probe outside the measured target and keeps the exact target kernel keys cold. Each timing is the median of three fresh Spark processes.rtcx::compilemssingle_chain1 / 2multi_shared_add32 / 65;28 / 57mixed_waves32 / 32;8 / 8;20 / 20rtcx::compileis the host wall time of NVRTC program creation, compilation, compile-log handling, CUBIN extraction, and program destruction on actual compiler cache misses. It excludes Row IR/source generation, cache lookup and waiting, library loading, and kernel execution.Plugin unique opsuses cudf-spark planner equivalence and is not a final libcudf Row IR node count. Per-program medians are calculated independently, so they do not necessarily sum to the median total.The estimated cache-miss overhead is
JIT cold - exact disk-cache first. The estimates at 20 million rows are 1,968.051 ms, 6,080.500 ms, and 5,403.616 ms; at 200 million rows they are 2,058.447 ms, 6,035.906 ms, and 5,344.617 ms. Every estimate changes by less than 5% after increasing the row count by 10x, indicating that source-JIT cache-miss overhead is primarily a fixed per-expression-shape cost.Cache-state controls and raw measurements
single_chainmulti_shared_addmixed_wavessingle_chainmulti_shared_addmixed_wavesEvery empty-cache sample uses a new
LIBCUDF_KERNEL_CACHE_PATHwithLIBCUDF_JIT_DISABLE_CUDA_CACHE=1.PCH warmfirst compiles a distinct one-row probe while leaving the target kernel key cold; NVRTC logs confirmed PCH reuse by every target kernel.Exact disk cachestarts a fresh process with the target kernel already present, whileExact in-process hotimmediately repeats the target action in the PCH process. Spark startup, the non-JIT bootstrap, PCH probe, and disk-cache population are excluded from measured target latency.Historical Row IR/LTO POC. An earlier proof of concept precompiled
INT32/INT64ADDandMULfragments and linked them with a thin runtime topology. For a 100K-row depth-4 expression, isolated fresh-process/fresh-cache first-call latency fell from approximately 2.33 seconds with a semantically equivalent source-JIT control to 0.12 seconds with LTO, saving approximately 2.21 seconds (95%, 20.2x). Hot kernel throughput was unchanged: the 100M-row, 30-iteration trimmed means were 4.096 ms for LTO and 4.098 ms for source JIT. This is POC evidence rather than a measurement of the current cudf-spark path; the source control used an outerIDENTITYnode to force source-JIT fallback. The prototype is available onproject-ast-jit-ultimate-perf.Checklists
Documentation
Testing
Performance