feat: feature-gate sqllogictests datafusion-substrait behind optional 'substrait' feature

[zhuqi-lucas]

Which issue does this PR close?

• Closes Feature-gate datafusion-substrait behind optional feature to reduce compile time #21269

Rationale for this change

datafusion-substrait is a heavyweight dependency that most developers and users don't need. Making it optional saves compile time for the common case.

What changes are included in this PR?

• datafusion-substrait is now an optional dependency in datafusion-sqllogictest/Cargo.toml
• New substrait feature flag gates the dependency
• #[cfg(feature = "substrait")] gates the module, imports, and run_test_file_substrait_round_trip function
• No-op fallback function returns an error when the feature is disabled
• CI workflow updated to pass --features substrait for the round-trip test

Are these changes tested?

• Compiles without substrait feature (default)
• Compiles with --features substrait
• CI substrait round-trip test updated to enable the feature

Are there any user-facing changes?

No — this only affects the sqllogictest binary, not the datafusion library itself. The substrait round-trip test requires --features substrait to run.

[Copilot]

Pull request overview

This PR makes the datafusion-substrait integration in the datafusion-sqllogictest crate opt-in behind a new substrait feature flag to reduce compile time and dependency weight for the default build.

Changes:

• Make datafusion-substrait an optional dependency and add a substrait feature in datafusion/sqllogictest/Cargo.toml.
• Gate the Substrait round-trip engine module and public re-exports behind #[cfg(feature = "substrait")].
• Add a non-substrait fallback for the Substrait round-trip runner and update CI to enable the feature when running that mode.

Reviewed changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
datafusion/sqllogictest/src/lib.rs	Gate the public re-export of DataFusionSubstraitRoundTrip behind the substrait feature.
datafusion/sqllogictest/src/engines/mod.rs	Gate the Substrait round-trip engine module and re-export behind the substrait feature.
datafusion/sqllogictest/Cargo.toml	Make datafusion-substrait optional and introduce a substrait feature enabling it.
datafusion/sqllogictest/bin/sqllogictests.rs	Conditionally import/use the Substrait engine; provide a runtime error fallback when feature is disabled.
.github/workflows/rust.yml	Update Substrait round-trip CI job to pass --features substrait.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[zhuqi-lucas]

CI failure in verify benchmark results is a pre-existing issue on main — explain.slt JSON format mismatch, unrelated to this PR (we only changed substrait feature gating).

[alamb]

The benchmark results result seems to pass on main; https://github.com/apache/datafusion/actions/runs/23868036699/job/69592615386

I'll update this branch to try and get a clean CI run

[alamb]

Makes sense to me -- thank yoU @zhuqi-lucas

FYI @gabotechs

I also verified that I can still run my core testing flow

nice cargo test --profile=ci --test sqllogictests

Locally and it passes

Only when I tried to run the substrait roundtrip tests did it fail (as expected)

andrewlamb@Andrews-MacBook-Pro-3:~/Software/datafusion2$ nice cargo test --profile=ci --test sqllogictests -- --substrait-round-trip
    Finished `ci` profile [unoptimized] target(s) in 0.21s
     Running bin/sqllogictests.rs (target/ci/deps/sqllogictests-703664fca1f0af0f)
Running with 16 test threads (available parallelism: 16)
Completed 416 test files in 0 seconds                                                                                             Execution error: Cannot run substrait round-trip: the 'substrait' feature is not enabled
Execution error: Cannot run substrait round-trip: the 'substrait' feature is not enabled
Execution error: Cannot run substrait round-trip: the 'substrait' feature is not enabled
Execution error: Cannot run substrait round-trip: the 'substrait' feature is not enabled
Execution error: Cannot run substrait round-trip: the 'substrait' feature is not enabled

[zhuqi-lucas]

The CI all green now, thanks @alamb for review.

Merged to main now.