fix: respect batch size in ordered aggregates#3
Merged
2010YOUY01 merged 1 commit intoJul 1, 2026
Merged
Conversation
alamb
force-pushed
the
codex/ordered-aggregate-batch-size
branch
from
June 30, 2026 17:52
b5dd963 to
c405cd7
Compare
alamb
force-pushed
the
codex/ordered-aggregate-batch-size
branch
from
June 30, 2026 17:54
c405cd7 to
8bb64f0
Compare
alamb
commented
Jun 30, 2026
| .await | ||
| .unwrap(); | ||
| assert!(collected_running.len() > 2); | ||
| // Running should produce more chunk than the usual AggregateExec. |
| impl<AggrMode> OrderedAggregateTable<AggrMode> { | ||
| #[expect( | ||
| clippy::too_many_arguments, | ||
| reason = "keeps ordered partial and final table construction explicit" |
Author
There was a problem hiding this comment.
I think we can make a builder style API for this, but for now just clippy ignore
alamb
marked this pull request as ready for review
June 30, 2026 18:10
alamb
pushed a commit
that referenced
this pull request
Jul 15, 2026
## Which issue does this PR close? - Part of apache#23334. > The numbers below come from the committed criterion benchmark added in apache#23335 (`cargo bench --bench array_has`) — **origin** = the per-row `eq` kernel (unoptimized `main` / apache#23335), **now** = with this optimization applied. Run the bench on `main` and on this branch to reproduce. Full disclosure - this was heavily assisted by AI, and I did my best to understand and justify every change here before submitting. ## Rationale for this change `array_has(array, element)` returns, for each row, whether the array contains the element. When the `element` (needle) is an array rather than a scalar, the needle argument is a column with one value per row, e.g. `array_has(t1.tags, t2.key)` in a join filter, execution goes through `array_has_dispatch_for_array` (the `ColumnarValue::Array` needle branch), which compared each row by invoking the Arrow `eq` kernel once per row. That kernel allocates a `BooleanArray` and pays downcast and dispatch overhead on every row. (The scalar-needle branch was optimized separately in apache#20374.) What this removes is the fixed per-row kernel overhead, not the element comparison itself, so the gain is largest for short lists and shrinks as lists grow. All numbers below are from the committed criterion benchmark (`cargo bench --bench array_has`, groups `array_has_array_null_patterns` / `array_has_array_by_size` / `array_has_array_by_rows`): the `array_has` UDF evaluated in isolation with an array needle, **origin** (the per-row `eq` kernel) vs **now**. "list length" is the number of elements in each row's array (not the row count). Not end-to-end query time. ### By data type and null pattern (list length 64, 10K rows) | element | element len | null pattern | origin | now | speedup | |-----------|----------------|----------------------|---------|---------|---------| | i64 | - | no nulls, found | 1.10 ms | 73 µs | 15.1x | | i64 | - | no nulls, not found | 1.07 ms | 72 µs. | 14.9x | | i64 | - | 30% nulls, found | 1.17 ms | 315 µs | 3.7x | | i64 | - | 30% nulls, not found | 1.10 ms | 274 µs | 4.0x | | i64 | - | all null | 1.10 ms | 272 µs | 4.0x | | i64 | - | collision | 1.10 ms | 270 µs | 4.1x | | Utf8 | short (inline) | no nulls | 2.57 ms | 1.01 ms | 2.5x | | Utf8 | short (inline) | 30% nulls | 3.37 ms | 1.52 ms | 2.2x | | Utf8 | long (>12B) | no nulls | 2.61 ms | 1.04 ms | 2.5x | | Utf8 | long (>12B) | 30% nulls | 3.31 ms | 1.52 ms | 2.2x | | Utf8 | - | all null | 1.26 ms | 256 µs | 4.9x | | LargeUtf8 | short (inline) | no nulls | 2.56 ms | 1.02 ms | 2.5x | | LargeUtf8 | short (inline) | 30% nulls | 3.20 ms | 1.54 ms | 2.1x | | LargeUtf8 | long (>12B) | no nulls | 2.67 ms | 1.05 ms | 2.6x | | LargeUtf8 | long (>12B) | 30% nulls | 3.42 ms | 1.59 ms | 2.2x | | LargeUtf8 | - | all null | 1.31 ms | 263 µs | 5.0x | | Utf8View | short (inline) | no nulls | 1.18 ms | 239 µs | 4.9x | | Utf8View | short (inline) | 30% nulls | 1.26 ms | 246 µs | 5.1x | | Utf8View | long (>12B) | no nulls | 2.86 ms | 1.17 ms | 2.4x | | Utf8View | long (>12B) | 30% nulls | 3.51 ms | 1.66 ms | 2.1x | | Utf8View | - | all null | 1.20 ms | 267 µs | 4.5x | The i64 null cases are uniform (~4x) whether the match is present, absent, the whole list is null, or the needle collides with a null slot's backing fill value — validity is folded in with one word-parallel op, so there is no per-row rescan and no null slot can match. Strings win ~2.1–2.5x mainly by dropping the per-row `BooleanArray` allocation. `Utf8View` additionally uses a view-aware compare: the byte length and 4-byte prefix packed into the 128-bit view reject non-matches before touching the data buffer, and an inline value (≤ 12 bytes) is matched by whole-view equality with no materialization at all — hence ~5x on short/inline strings. When long strings share a prefix (e.g. ARNs) the prefix can't reject, so `Utf8View` falls in line with the other string types (~2.1–2.4x). No string case regresses. ### By list length (i64, 30% element nulls, not found, 10K rows) | elems/row | origin | now | speedup | |-----------|---------|---------|--------------------------------------| | 8 | 1.03 ms | 111 µs | 9.3x | | 32 | 1.07 ms | 197 µs | 5.5x | | 128 | 1.18 ms | 446 µs | 2.6x | | 256 | 1.28 ms | 780 µs | 1.6x | | 512 | 1.54 ms | 1.44 ms | 1.1x | | 1024 | 2.17 ms | 2.15 ms | 1.0x (falls back to per-row kernel) | The element-null branch makes a few passes over the values; past a moderate average list length (`NULL_FAST_PATH_MAX_LEN`) the per-row kernel wins, so it bails to it there — no meaningful regression. That average is measured over the visible (sliced) region, so a sliced array's hidden child elements can't route a small window to the slow path. The all-valid fold has no such crossover. ### By row count (i64, 8 elems/row, 30% nulls, not found) | rows | origin | now | speedup | |------|-----------|----------|---------| | 10K | 1.04 ms | 111 µs | 9.4x | | 100K | 10.42 ms | 1.09 ms | 9.6x | | 1M | 102.68 ms | 10.91 ms | 9.4x | Invariant to the number of rows — the per-row overhead removed is a fixed cost, so absolute savings scale linearly with the column height. The remaining benchmarks in the suite (scalar `array_has`, `array_has_all`, `array_has_any` — paths this PR does not touch) are unchanged (median 0.99x, within measurement noise), confirming no regression outside the array-needle path. ### End-to-end (context) For a query dominated by an array-needle `array_has` join filter (a `NestedLoopJoinExec` with `filter=array_has(tags, key)` over 3000x3000 rows of 8-element lists) total time drops from 0.95s to 0.059s (~16x, identical results). For a workload where `array_has` is a smaller fraction, e.g. the ~6% of profile that motivated this (see apache#18070 / apache#18161, which fixed the join's deep-copy but left the per-row `array_has` cost), the overall speedup is single-digit percent. ## What changes are included in this PR? A fast path for primitive and string element types in `array_has_dispatch_for_array`, preserving the Arrow `eq` kernel semantics (total-order float equality; null elements never match): - **All-valid elements:** each row is a single branchless OR-reduction over the raw native value slice (auto-vectorizes; the common case). - **Element nulls:** a null slot's backing value is arbitrary, so the per-element equality bitmap is ANDed with the validity bitmap (one word-parallel op, no per-element branch) before reducing each row to "any bit set", a null slot can never match regardless of its value. This branch is processed in row chunks so the scratch buffer stays bounded, and past `NULL_FAST_PATH_MAX_LEN` average elements/row a length check over the visible (sliced) region bails to the per-row kernel (see the list-length table). - **String elements:** each row is a single pass over the row's values (compare, then consult validity only on a match). `Utf8View` compares the packed 128-bit views directly — length + 4-byte prefix reject non-matches before any data-buffer access, and an inline value (≤ 12 bytes) matches by whole-view equality with no materialization. - **Nested (and any other) element types** keep using the per-row `eq` kernel. The array-needle benchmarks used for the numbers above are added in #3 (null patterns, list length, and row count). ## Are these changes tested? Yes: - New unit tests for the array-needle path covering element nulls, the null-fill collision (needle equal to a null slot's backing value), total-order float equality (`NaN` / `-0.0`), sliced arrays (including a small visible window over a large backing child), `LargeList` offsets, empty rows, a multi-chunk input, and a long-list input that exercises the per-row fallback, each cross-checked against the original per-row `eq` kernel as an oracle. - Existing `array_has` / `array_contains` / `join_lists` sqllogictest suites pass. ## Are there any user-facing changes? No. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed fix for this (see comments)