Skip to content

Commit 95398f0

Browse files
authored
Test: add more aggregation focused dictionary sql logic test (#23280)
## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes #123` indicates that this PR will close issue #123. --> works towards #22682. ## Rationale for this change There is a lack of testing for multi-dictionary group bys. It make sense to introduce these test before the implementation of `Dict<K,V>` in #23187 <!-- Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed. Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes. --> ## What changes are included in this PR? introduces a couple test - grouping by `Dict<_,largeutf8>` - mixing grouping by dictionarys and non dictionary columns - a 3-way group by where each column is `dict<_,_>` - the test also have nulls sprinkled in to verify null handling <!-- There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR. --> ## Are these changes tested? the changes are test. <!-- We typically require tests for all PRs in order to: 1. Prevent the code from being accidentally broken by subsequent changes 2. Serve as another way to document the expected behavior of the code If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? --> ## Are there any user-facing changes? no <!-- If there are user-facing changes then we may require documentation to be updated before approving the PR. --> <!-- If there are any breaking changes to public APIs, please add the `api change` label. -->
1 parent 479a0aa commit 95398f0

1 file changed

Lines changed: 118 additions & 0 deletions

File tree

datafusion/sqllogictest/test_files/dictionary.slt

Lines changed: 118 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -515,3 +515,121 @@ DROP TABLE dict_hash_10;
515515

516516
statement ok
517517
DROP TABLE dict_hash_src;
518+
519+
statement ok
520+
CREATE TABLE dict_large_utf8 AS
521+
SELECT
522+
arrow_cast(column1, 'Dictionary(Int32, LargeUtf8)') AS tag,
523+
arrow_cast(column2, 'Float64') AS val
524+
FROM (VALUES ('alpha', 1.0), ('beta', 2.0), ('alpha', 3.0), ('gamma', 4.0), ('beta', 5.0), (NULL, 6.0));
525+
526+
query TRI rowsort
527+
SELECT tag, SUM(val), COUNT(*) FROM dict_large_utf8 GROUP BY tag;
528+
----
529+
NULL 6 1
530+
alpha 4 2
531+
beta 7 2
532+
gamma 4 1
533+
534+
statement ok
535+
DROP TABLE dict_large_utf8;
536+
537+
# multiple dictionary columns as group keys
538+
539+
statement ok
540+
CREATE TABLE dict_multi_key AS
541+
SELECT
542+
arrow_cast(column1, 'Dictionary(Int32, Utf8)') AS region,
543+
arrow_cast(column2, 'Dictionary(Int16, Utf8)') AS status,
544+
arrow_cast(column3, 'Dictionary(Int8, Utf8)') AS tier,
545+
arrow_cast(column4, 'Float64') AS amount
546+
FROM (
547+
VALUES
548+
('us', 'active', 'gold', 100.0),
549+
('eu', 'active', 'silver', 200.0),
550+
('us', 'active', 'gold', 300.0),
551+
('eu', 'inactive', 'gold', 400.0),
552+
('us', 'inactive', 'silver', 500.0),
553+
('eu', 'inactive', 'gold', 600.0),
554+
('us', 'active', 'silver', 150.0),
555+
('eu', 'active', 'silver', 250.0),
556+
(NULL, 'active', 'gold', 700.0)
557+
);
558+
559+
query TTTRI rowsort
560+
SELECT region, status, tier, SUM(amount), COUNT(*) FROM dict_multi_key GROUP BY region, status, tier;
561+
----
562+
NULL active gold 700 1
563+
eu active silver 450 2
564+
eu inactive gold 1000 2
565+
us active gold 400 2
566+
us active silver 150 1
567+
us inactive silver 500 1
568+
569+
statement ok
570+
DROP TABLE dict_multi_key;
571+
572+
# mixed dict (2) and non-dict (2) group keys with nulls spread across all columns
573+
574+
statement ok
575+
CREATE TABLE dict_mixed_nulls AS
576+
SELECT
577+
arrow_cast(column1, 'Dictionary(Int32, Utf8)') AS region,
578+
arrow_cast(column2, 'Dictionary(Int16, Utf8)') AS status,
579+
arrow_cast(column3, 'Utf8') AS tier,
580+
arrow_cast(column4, 'Int32') AS category,
581+
arrow_cast(column5, 'Float64') AS amount
582+
FROM (
583+
VALUES
584+
('us', 'active', 'gold', 1, 100.0),
585+
('us', 'active', 'gold', 1, 200.0),
586+
('eu', 'active', 'silver', 2, 300.0),
587+
(NULL, 'active', 'gold', 1, 400.0),
588+
('us', NULL, 'gold', 1, 500.0),
589+
('us', 'active', NULL, 1, 600.0),
590+
('us', 'active', 'gold', NULL, 700.0)
591+
);
592+
593+
query TTTIRI rowsort
594+
SELECT region, status, tier, category, SUM(amount), COUNT(*) FROM dict_mixed_nulls GROUP BY region, status, tier, category;
595+
----
596+
NULL active gold 1 400 1
597+
eu active silver 2 300 1
598+
us NULL gold 1 500 1
599+
us active NULL 1 600 1
600+
us active gold 1 300 2
601+
us active gold NULL 700 1
602+
603+
statement ok
604+
DROP TABLE dict_mixed_nulls;
605+
606+
##########
607+
## Aggregation tests: COUNT(DISTINCT) on dictionary columns
608+
##########
609+
610+
statement ok
611+
CREATE TABLE dict_count_distinct AS
612+
SELECT
613+
arrow_cast(column1, 'Dictionary(Int64, Utf8)') AS region,
614+
arrow_cast(column2, 'Dictionary(Int8, Utf8)') AS sensor
615+
FROM (
616+
VALUES
617+
('north', 's1'), ('north', 's2'), ('north', 's1'), ('north', 's3'),
618+
('south', 's3'), ('south', 's3'), ('south', 's4'),
619+
('east', 's1'),
620+
(NULL, 's5'),
621+
('north', NULL)
622+
);
623+
624+
query TI rowsort
625+
SELECT region, COUNT(DISTINCT sensor) FROM dict_count_distinct GROUP BY region;
626+
----
627+
NULL 1
628+
east 1
629+
north 3
630+
south 2
631+
632+
statement ok
633+
DROP TABLE dict_count_distinct;
634+
635+

0 commit comments

Comments
 (0)