Skip to content

Commit 9f3c0ff

Browse files
committed
MDEV-24943: Implement FILTER clause support for aggregate functions
Aggregates lacked the SQL-standard FILTER clause, forcing CASE-based workarounds that reduced readability across (sum, avg, count, …). This update introduces the ability to specify a FILTER clause for aggregate functions, allowing for more granular control over which rows are included in the aggregation. Also, improves standards compliance and makes queries clearer and more readable. The FILTER(WHERE ...) condition may contain any expression allowed in regular WHERE clauses, except subqueries, window functions, and outer references.
1 parent 101346c commit 9f3c0ff

File tree

9 files changed

+1247
-37
lines changed

9 files changed

+1247
-37
lines changed
Lines changed: 329 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,329 @@
1+
#
2+
# Setup test table
3+
#
4+
#
5+
# Basic aggregates with WHERE
6+
#
7+
avg_result sum_result count_result min_result max_result group_concat_result bit_and_result bit_or_result bit_xor_result json_arrayagg_result json_objectagg_result std_result stddev_result stddev_pop_result stddev_samp_result variance_result var_pop_result var_samp_result
8+
137.5000 550 3 75 200 Novel 0 15 6 ["Phone","Tablet",null,null,"Laptop",null,null,null] {"item2":"Tablet", "item5":"Laptop"} 44.7214 41.4578 0.0000 54.1987 1100.0000 4000.0000 312.5000
9+
#
10+
# Aggregates with DISTINCT and WHERE
11+
#
12+
avg_result sum_result count_result min_result max_result group_concat_result
13+
175.0000 525 1 75 200 Laptop,Phone,Tablet
14+
#
15+
# Aggregates with complex WHERE conditions (AND)
16+
#
17+
avg_result sum_result count_result min_result max_result group_concat_result bit_and_result bit_or_result bit_xor_result json_arrayagg_result std_result variance_result
18+
183.3333 250 5 75 200 Laptop,Phone,Tablet 0 15 9 [null,"Tablet",null,null,"Laptop",null,null,null] 41.4578 742.1875
19+
#
20+
# Aggregates with WHERE IN and OR
21+
#
22+
avg_result sum_result count_result min_result max_result group_concat_result bit_and_result bit_or_result bit_xor_result json_arrayagg_result std_result variance_result
23+
137.5000 825 6 50 200 Guide,Novel,Pants,Phone,Shirt,Textbook 0 15 14 ["Phone","Tablet",null,null,"Laptop","Shirt","Pants",null] 55.0973 2447.9167
24+
#
25+
# Aggregates with WHERE BETWEEN
26+
#
27+
avg_result sum_result count_result min_result max_result group_concat_result bit_and_result bit_or_result bit_xor_result json_arrayagg_result std_result variance_result
28+
150.0000 625 5 75 150 Laptop,Pants,Phone,Shirt,Tablet 0 15 1 ["Phone","Tablet",null,"Textbook","Laptop",null,null,"Guide"] 20.4124 555.5556
29+
#
30+
# Aggregates with WHERE LIKE
31+
#
32+
avg_result sum_result count_result min_result max_result group_concat_result bit_and_result bit_or_result bit_xor_result json_arrayagg_result std_result variance_result
33+
131.2500 250 3 100 100 Guide,Novel,Textbook 6 15 5 [null,"Tablet",null,null,null,"Shirt",null,null] 12.5000 2447.9167
34+
#
35+
# Basic aggregates with GROUP BY
36+
#
37+
category avg_result sum_result count_result min_result max_result group_concat_result bit_and_result bit_or_result bit_xor_result json_arrayagg_result std_result variance_result
38+
Books 200.0000 400 2 NULL 200 NULL 7 15 8 [null,null,null] 0.0000 0.0000
39+
Clothing 75.0000 NULL 2 75 75 NULL 0 0 0 [null,null] NULL NULL
40+
Electronics 116.6667 150 3 100 100 Laptop,Phone,Tablet 0 15 1 [null,"Tablet","Laptop"] 23.5702 555.5556
41+
#
42+
# Aggregates with GROUP BY and WHERE IS NOT NULL
43+
#
44+
category avg_result sum_result count_result min_result max_result group_concat_result bit_and_result bit_or_result bit_xor_result json_arrayagg_result std_result variance_result
45+
Books 200.0000 400 2 200 200 Guide,Novel,Textbook 7 15 8 ["Novel","Textbook","Guide"] 0.0000 0.0000
46+
Clothing 62.5000 75 2 75 75 Pants,Shirt 0 15 15 ["Shirt","Pants"] 12.5000 156.2500
47+
Electronics 116.6667 350 3 100 150 Laptop,Phone,Tablet 0 15 1 ["Phone","Tablet","Laptop"] 23.5702 555.5556
48+
#
49+
# Aggregates with multiple GROUP BY columns
50+
#
51+
category status avg_result sum_result count_result min_result max_result group_concat_result bit_and_result bit_or_result bit_xor_result json_arrayagg_result std_result variance_result
52+
Books active 200.0000 400 2 200 200 Guide,Textbook 7 15 8 ["Textbook","Guide"] 0.0000 0.0000
53+
Books inactive NULL NULL 1 NULL NULL Novel 18446744073709551615 0 0 ["Novel"] NULL NULL
54+
Clothing NULL 50.0000 50 1 NULL 50 Shirt 3 0 3 ["Shirt"] 0.0000 0.0000
55+
Clothing active 75.0000 75 1 75 75 Pants 12 12 12 ["Pants"] 0.0000 0.0000
56+
Electronics active 116.6667 350 3 100 150 Laptop,Phone,Tablet 0 15 9 ["Phone","Tablet","Laptop"] 23.5702 555.5556
57+
#
58+
# GROUP BY with HAVING on aggregates
59+
#
60+
category avg_result sum_result count_result min_result max_result
61+
Books 200.0000 400 2 NULL 200
62+
#
63+
# HAVING with alias
64+
#
65+
category avg_value sum_value count_value min_value max_value
66+
Books 200.0000 400 2 NULL 200
67+
#
68+
# WHERE + GROUP BY + HAVING
69+
#
70+
category avg_result sum_result count_result min_result max_result
71+
Books 200.0000 400 2 NULL 200
72+
Electronics 116.6667 150 3 100 100
73+
#
74+
# WHERE + multiple aggregates with FILTER
75+
#
76+
category total_avg active_avg total_sum active_sum total_count active_count total_min active_min total_max active_max
77+
Books 200.0000 200.0000 400 NULL 2 2 NULL NULL 200 200
78+
Clothing 62.5000 75.0000 NULL NULL 2 2 NULL NULL 75 75
79+
Electronics 116.6667 116.6667 150 350 3 1 100 100 150 150
80+
#
81+
# WHERE + FILTER + HAVING
82+
#
83+
category active_avg active_sum active_count
84+
Books 200.0000 400 0
85+
Clothing 75.0000 NULL 0
86+
Electronics 116.6667 150 3
87+
#
88+
# WHERE + window function (OVER)
89+
#
90+
category cat_avg cat_sum cat_count cat_min cat_max
91+
Books 200.0000 400 2 NULL 200
92+
Books 200.0000 400 2 NULL 200
93+
Books 200.0000 400 2 NULL 200
94+
Clothing 75.0000 NULL 2 75 75
95+
Clothing 75.0000 NULL 2 75 75
96+
Electronics 116.6667 150 3 100 100
97+
Electronics 116.6667 150 3 100 100
98+
Electronics 116.6667 150 3 100 100
99+
#
100+
# WHERE + window function with ORDER BY
101+
#
102+
id running_avg running_sum running_count running_min running_max
103+
1 100.0000 100 1 100 100
104+
2 125.0000 250 2 100 150
105+
3 125.0000 250 3 100 150
106+
4 150.0000 450 4 100 200
107+
5 137.5000 550 5 100 200
108+
6 120.0000 550 6 100 200
109+
7 112.5000 625 7 75 200
110+
8 125.0000 825 8 75 200
111+
#
112+
# WHERE + window function with frame specification
113+
#
114+
id windowed_avg windowed_sum windowed_count windowed_min windowed_max
115+
1 125.0000 150 2 100 NULL
116+
2 125.0000 150 2 100 NULL
117+
3 175.0000 350 2 150 200
118+
4 150.0000 200 2 100 200
119+
5 150.0000 200 3 100 200
120+
6 87.5000 NULL 3 75 100
121+
7 137.5000 200 3 75 200
122+
8 137.5000 200 2 75 200
123+
#
124+
# Subquery with aggregates in WHERE
125+
#
126+
id category status value price amount name key_name value_col bit_value geom
127+
3 Books inactive NULL 19.99 19.99 Novel item3 Novel NULL POINT(3 3)
128+
4 Books active 200 29.99 29.99 Textbook item4 Textbook 7 POINT(4 4)
129+
8 Books active 200 24.99 24.99 Guide item8 Guide 15 POINT(8 8)
130+
#
131+
# Aggregates with GROUP BY and ORDER BY
132+
#
133+
category avg_val sum_val count_val min_val max_val
134+
Books 200.0000 400 2 NULL 200
135+
Electronics 116.6667 150 3 100 100
136+
Clothing 75.0000 NULL 2 75 75
137+
#
138+
# Complete query with WHERE, GROUP BY, HAVING, ORDER BY
139+
#
140+
category avg_val sum_val count_val min_val max_val
141+
Books 200.0000 400 2 NULL 200
142+
Electronics 116.6667 150 3 100 100
143+
#
144+
# Aggregates with GROUP BY, ORDER BY, and LIMIT
145+
#
146+
category avg_val sum_val count_val
147+
Books 200.0000 400 2
148+
Electronics 116.6667 150 3
149+
#
150+
# Aggregates with ROLLUP
151+
#
152+
category status avg_val sum_val count_val min_val max_val
153+
Books active 200.0000 400 2 NULL 200
154+
Books inactive NULL NULL 0 NULL NULL
155+
Books NULL 200.0000 400 2 NULL 200
156+
Clothing NULL NULL NULL 1 NULL 50
157+
Clothing active 75.0000 NULL 1 75 75
158+
Clothing NULL 75.0000 NULL 2 75 75
159+
Electronics active 116.6667 150 3 100 100
160+
Electronics NULL 116.6667 150 3 100 100
161+
NULL NULL 137.5000 550 7 75 200
162+
#
163+
# Most complete query with all clauses and aggregates
164+
#
165+
category status avg_value sum_value count_value min_value max_value avg_amount sum_amount distinct_value_avg distinct_value_sum distinct_value_count group_concat_result bit_and_result bit_or_result bit_xor_result json_arrayagg_result std_result variance_result
166+
Books active 200.0000 400 2 NULL 200 NULL 54.98 200.0000 200 1 Guide 7 15 0 [null,null] NULL 0.0000
167+
Electronics active 100.0000 350 1 100 150 333.326667 999.98 100.0000 NULL 2 Laptop 0 15 7 [null,null,null] 23.5702 555.5556
168+
Clothing active 75.0000 75 0 75 75 NULL NULL 75.0000 NULL 1 Pants 18446744073709551615 0 0 [null] 0.0000 NULL
169+
#
170+
# COUNT with column names (not just *)
171+
#
172+
count_value count_category count_name
173+
6 3 3
174+
#
175+
# COUNT DISTINCT with multiple columns
176+
#
177+
count_distinct
178+
3
179+
#
180+
# FILTER on indexed column
181+
#
182+
# Create index on status column
183+
avg_result sum_result count_result
184+
137.5000 350 3
185+
#
186+
# Empty result set (all rows filtered out)
187+
#
188+
avg_result sum_result count_result min_result max_result
189+
NULL NULL 0 NULL NULL
190+
#
191+
# FILTER with NULL conditions
192+
#
193+
avg_result count_result sum_result
194+
NULL 0 NULL
195+
#
196+
# FILTER with JOINs
197+
#
198+
# Create second table for JOIN test
199+
category avg_result sum_result count_result
200+
Books 200.0000 200 2
201+
Electronics 150.0000 NULL 0
202+
#
203+
# FILTER with window functions - different partitions
204+
#
205+
category status cat_avg status_avg cat_count status_count
206+
Books active 200.0000 183.3333 2 4
207+
Books active 200.0000 183.3333 2 4
208+
Books inactive 200.0000 NULL 2 0
209+
Clothing NULL 75.0000 NULL 2 0
210+
Clothing active 75.0000 183.3333 2 4
211+
Electronics active 116.6667 183.3333 3 4
212+
Electronics active 116.6667 183.3333 3 4
213+
Electronics active 116.6667 183.3333 3 4
214+
#
215+
# FILTER with empty table
216+
#
217+
# Create empty table
218+
avg_result sum_result count_result min_result max_result
219+
NULL NULL 0 NULL NULL
220+
#
221+
# FILTER with CASE in WHERE clause
222+
#
223+
avg_result sum_result count_result
224+
137.5000 550 3
225+
#
226+
# ORDER BY with FILTER aggregates
227+
#
228+
category avg_val sum_val
229+
Books 200.0000 400
230+
Electronics 116.6667 150
231+
Clothing 75.0000 NULL
232+
#
233+
# Aggregates on id column itself with FILTER
234+
#
235+
avg_id sum_id count_id min_id max_id distinct_count_id
236+
4.5000 14 3 1 8 5
237+
#
238+
# Aggregates on id with GROUP BY and FILTER
239+
#
240+
category avg_id sum_id count_id min_id max_id
241+
Books 6.0000 12 2 NULL 8
242+
Clothing 6.5000 7 0 7 7
243+
Electronics 5.0000 8 1 1 5
244+
#
245+
# ========================================
246+
# ERROR CASES SECTION
247+
# ========================================
248+
#
249+
#
250+
# FILTER with Non-Aggregate Window Functions (Ranking)
251+
#
252+
# FILTER with RANK (should error - FILTER only works with aggregates)
253+
ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'FILTER (WHERE status = 'active') OVER (ORDER BY value) FROM test_aggregates' at line 1
254+
#
255+
# FILTER with Non-Aggregate Window Functions (Navigation)
256+
#
257+
# FILTER with LAG (should error - FILTER only works with aggregates)
258+
ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'FILTER (WHERE status = 'active') OVER (ORDER BY id) FROM test_aggregates' at line 1
259+
#
260+
# Wrong Order - OVER before FILTER
261+
#
262+
# OVER clause before FILTER clause (should error - correct order is FILTER then OVER)
263+
ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '(WHERE status = 'active') FROM test_aggregates' at line 1
264+
#
265+
# Non-Aggregate Scalar Functions
266+
#
267+
# Regular scalar function (should error - FILTER only for aggregates)
268+
ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '(WHERE status = 'active') FROM test_aggregates' at line 1
269+
#
270+
# Plain Column or Expression
271+
#
272+
# FILTER on plain column (should error - not a function)
273+
ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '(WHERE status = 'active') FROM test_aggregates' at line 1
274+
#
275+
# Empty or Invalid Clause
276+
#
277+
# Empty FILTER clause (should error - WHERE condition required)
278+
ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ') FROM test_aggregates' at line 1
279+
#
280+
# Nested FILTER Clauses
281+
#
282+
# Nested FILTER clauses (should error - cannot nest FILTER)
283+
ERROR HY000: Invalid use of group function
284+
#
285+
# Invalid WHERE Syntax
286+
#
287+
# Multiple WHERE keywords (should error)
288+
ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'WHERE value > 100) FROM test_aggregates' at line 1
289+
#
290+
# Column Aliases
291+
#
292+
# Using column alias in FILTER WHERE (should error - alias not available)
293+
ERROR 42S22: Unknown column 'val' in 'SELECT'
294+
#
295+
# Subqueries in WHERE Condition
296+
#
297+
# Scalar subquery in FILTER WHERE (should error - subqueries not allowed)
298+
ERROR HY000: Incorrect usage of subqueries in FILTER clause and
299+
# IN subquery in FILTER WHERE (should error - subqueries not allowed)
300+
ERROR HY000: Incorrect usage of subqueries in FILTER clause and Item_in_subselect::fix_fields
301+
#
302+
# FILTER with Window Functions in WHERE Condition
303+
#
304+
# Window function in FILTER WHERE condition (should error - not allowed)
305+
ERROR HY000: Incorrect usage of window function and FILTER clause
306+
#
307+
# FILTER with aggregate functions in WHERE clause (should expect error)
308+
#
309+
# This should error - aggregate in WHERE clause
310+
ERROR HY000: Invalid use of group function
311+
#
312+
# FILTER with error cases - syntax errors
313+
#
314+
# Missing WHERE keyword (should error)
315+
ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'status = 'active') FROM test_aggregates' at line 1
316+
# Missing parentheses (should error)
317+
ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'WHERE status = 'active' FROM test_aggregates' at line 1
318+
#
319+
# Illegal use of FILTER with non-aggregate UDFs
320+
#
321+
# Create a non-aggregate UDF for testing
322+
# FILTER clause with non-aggregate UDF should fail
323+
ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '(WHERE status = 'active') FROM test_aggregates' at line 1
324+
#
325+
# CLEANUP
326+
#
327+
#
328+
# End of aggregates FILTER test
329+
#

0 commit comments

Comments
 (0)