Skip to content

Adaptive (AQE) execution drops all rows for TPC-DS q61 at target_partitions=1 #2185

Description

@andygrove

Describe the bug

With adaptive execution enabled (ballista.planner.adaptive.enabled=true) at target_partitions=1, TPC-DS q61 returns an empty result where single-process DataFusion returns one row:

Query 61 VERIFY MISMATCH: Execution error: result mismatch: expected 1 rows, got 0 rows

Surfaced by extending the TPC-DS SF1 correctness gate to cover the adaptive planner and single-partition execution (#2182).

Scope

To Reproduce

Generate TPC-DS SF1 Parquet with tpcgen-cli, start a scheduler and executor, then:

tpcds --host localhost --port 50050 --path <data> --partitions 1 --verify \
  --query 61 \
  -c datafusion.optimizer.prefer_hash_join=false \
  -c ballista.planner.adaptive.enabled=true
# -> Query 61 VERIFY MISMATCH: expected 1 rows, got 0 rows

Passes with --partitions 16, and with --partitions 1 when the adaptive planner is off.

Expected behavior

The adaptive planner should return the same single row as single-process DataFusion, at every partition count.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions