Skip to content

Document native UDF ABI alignment requirements#629

Merged
nvliyuan merged 1 commit into
NVIDIA:mainfrom
nvliyuan:docs-native-udf-abi-compat
May 28, 2026
Merged

Document native UDF ABI alignment requirements#629
nvliyuan merged 1 commit into
NVIDIA:mainfrom
nvliyuan:docs-native-udf-abi-compat

Conversation

@nvliyuan

Copy link
Copy Markdown
Collaborator

Summary

  • Document that native UDF builds must align cuDF/RMM/CCCL headers with the libcudf.so packaged in the rapids-4-spark jar.
  • Call out undefined symbols and native crashes as symptoms of ABI drift, especially for snapshot or locally built jars.
  • Recommend rebuilding native UDFs after jar changes and removing cached native dependencies before rebuilds.

Closes #628

Test plan

  • Documentation-only change; verified README diff locally.
  • Checked Cursor lints for the edited README.

@greptile-apps

greptile-apps Bot commented May 28, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds a "Native ABI compatibility note" paragraph to the fast-build section of the UDF README, documenting that native UDFs must be compiled against cuDF/RMM/CCCL headers that match the libcudf.so bundled in the rapids-4-spark jar, and prescribing concrete remediation steps.

  • Documents the ABI drift failure modes (undefined symbols, native crashes) that arise when headers used at compile time do not match the jar's libcudf.so, with extra guidance for snapshot and locally built jars.
  • Instructs users to remove target/native-deps and target/cudf-repo before rebuilding after any version or branch change, preventing stale headers or cached libraries from silently causing mismatches.

Confidence Score: 5/5

Documentation-only change; no runtime code is modified, so there is no risk of behavioral regression.

The change is a single prose addition to a README with no code, build scripts, or configuration files touched. The new text is technically accurate about ABI drift, names the correct cache directories to clear, and is placed logically within the existing fast-build section.

No files require special attention.

Important Files Changed

Filename Overview
examples/UDF-Examples/RAPIDS-accelerated-UDFs/README.md Adds an 18-line 'Native ABI compatibility note' section explaining header/library alignment requirements and cache-clearing steps for native UDF builds.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Change rapids-4-spark jar version] --> B{Headers match libcudf.so in jar?}
    B -- Yes --> C[Build succeeds, UDF loads correctly]
    B -- No --> D[ABI drift: undefined symbols or native crash]
    A --> E[Update cudf.git.branch, rapids-cmake branch, RMM/CCCL versions]
    E --> F[Remove target/native-deps and target/cudf-repo]
    F --> G[mvn clean package -Pudf-native-examples]
    G --> B
Loading

Reviews (2): Last reviewed commit: "Document native UDF ABI alignment" | Re-trigger Greptile

Signed-off-by: liyuan <yuali@nvidia.com>
@nvliyuan nvliyuan force-pushed the docs-native-udf-abi-compat branch from 34680bc to 50f9289 Compare May 28, 2026 06:44
@nvliyuan nvliyuan requested a review from thirtiseven May 28, 2026 06:49

@thirtiseven thirtiseven left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@nvliyuan nvliyuan merged commit 1821629 into NVIDIA:main May 28, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Native UDF build does not guarantee ABI match between extracted libcudf.so and downloaded headers

3 participants