Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](Planner) fix lateral view when use cte as input table #44561

Draft
wants to merge 1 commit into
base: branch-2.0
Choose a base branch
from

Conversation

LiBinfeng-01
Copy link
Collaborator

@LiBinfeng-01 LiBinfeng-01 commented Nov 25, 2024

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:
When using explode with cte as input, it would failed when analyze select list in Planner
Cause by missing lateral view message when copy lateralViewRef
Fix this problem by filling lateral view message to copied lateralViewRef

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@github-actions github-actions bot added area/planner Issues or PRs related to the query planner kind/test labels Nov 25, 2024
@LiBinfeng-01 LiBinfeng-01 marked this pull request as draft November 25, 2024 09:48
@LiBinfeng-01
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49461 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f1937a16b23224f451bba79d6fab47f416130a9c, data reload: false

------ Round 1 ----------------------------------
q1	17768	4456	4375	4375
q2	2069	158	145	145
q3	10396	1880	1919	1880
q4	10273	1229	1484	1229
q5	8584	3974	3910	3910
q6	238	122	128	122
q7	2001	1606	1580	1580
q8	9324	2755	2717	2717
q9	10546	10095	10031	10031
q10	8665	3517	3578	3517
q11	429	248	249	248
q12	473	295	297	295
q13	18366	3984	4051	3984
q14	355	335	330	330
q15	519	450	468	450
q16	540	455	465	455
q17	1139	969	935	935
q18	7279	6917	6956	6917
q19	1678	1587	1531	1531
q20	507	325	294	294
q21	4478	4143	4121	4121
q22	510	395	406	395
Total cold run time: 116137 ms
Total hot run time: 49461 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4325	4314	4281	4281
q2	321	220	227	220
q3	4141	4130	4179	4130
q4	2780	2751	2754	2751
q5	7241	7202	7141	7141
q6	237	121	120	120
q7	3254	2871	2855	2855
q8	4367	4519	4531	4519
q9	14265	13823	13907	13823
q10	4257	4258	4247	4247
q11	759	677	678	677
q12	1025	874	855	855
q13	6569	3752	3737	3737
q14	454	437	421	421
q15	498	445	443	443
q16	622	597	567	567
q17	3812	3829	3878	3829
q18	8861	8763	8819	8763
q19	1704	1667	1671	1667
q20	2432	2122	2136	2122
q21	8447	8469	8441	8441
q22	1110	957	983	957
Total cold run time: 81481 ms
Total hot run time: 76566 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 212953 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f1937a16b23224f451bba79d6fab47f416130a9c, data reload: false

query1	945	395	416	395
query2	6556	2162	1967	1967
query3	6928	208	206	206
query4	23026	21570	21483	21483
query5	19724	6495	6525	6495
query6	293	221	241	221
query7	4342	310	301	301
query8	245	266	250	250
query9	3136	2682	2610	2610
query10	476	319	308	308
query11	15670	15163	14930	14930
query12	125	79	79	79
query13	1031	439	448	439
query14	17507	13623	13632	13623
query15	378	221	238	221
query16	6465	276	271	271
query17	1783	945	921	921
query18	891	323	309	309
query19	207	156	154	154
query20	112	101	103	101
query21	194	100	97	97
query22	5221	5010	4978	4978
query23	34105	33551	33560	33551
query24	6884	6303	6347	6303
query25	533	419	440	419
query26	987	169	167	167
query27	2296	296	293	293
query28	6132	2316	2242	2242
query29	2813	2789	2680	2680
query30	245	169	163	163
query31	938	750	767	750
query32	72	76	61	61
query33	442	275	266	266
query34	868	473	481	473
query35	1132	942	931	931
query36	1499	1318	1187	1187
query37	95	61	59	59
query38	3091	2955	2975	2955
query39	1397	1348	1339	1339
query40	205	96	97	96
query41	41	39	49	39
query42	94	95	86	86
query43	672	587	679	587
query44	1155	712	724	712
query45	248	229	233	229
query46	1245	966	969	966
query47	1878	1861	1860	1860
query48	509	419	417	417
query49	631	377	378	377
query50	874	626	647	626
query51	4783	4653	4755	4653
query52	99	85	86	85
query53	226	179	192	179
query54	2675	2460	2497	2460
query55	105	88	89	88
query56	205	191	206	191
query57	1253	1163	1125	1125
query58	227	225	196	196
query59	3567	3326	3254	3254
query60	227	209	197	197
query61	97	93	108	93
query62	851	435	524	435
query63	205	177	175	175
query64	3285	1597	1431	1431
query65	3671	3541	3598	3541
query66	799	450	402	402
query67	15821	15107	16881	15107
query68	9476	642	645	642
query69	496	284	289	284
query70	1548	1546	1337	1337
query71	410	316	324	316
query72	6811	4888	4920	4888
query73	753	320	315	315
query74	6338	5925	5826	5826
query75	5243	3692	3717	3692
query76	5260	1096	1130	1096
query77	882	255	265	255
query78	13121	11657	12099	11657
query79	8910	655	649	649
query80	1356	392	386	386
query81	487	245	239	239
query82	1639	106	96	96
query83	168	133	140	133
query84	254	73	71	71
query85	891	332	319	319
query86	330	344	318	318
query87	3189	3002	3050	3002
query88	4682	2309	2299	2299
query89	503	304	343	304
query90	1949	215	220	215
query91	159	129	125	125
query92	60	55	53	53
query93	6743	572	560	560
query94	721	217	205	205
query95	1962	2037	2083	2037
query96	662	328	320	320
query97	6659	6423	6488	6423
query98	229	200	200	200
query99	2876	894	886	886
Total cold run time: 319199 ms
Total hot run time: 212953 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.27 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f1937a16b23224f451bba79d6fab47f416130a9c, data reload: false

query1	0.02	0.02	0.03
query2	0.07	0.03	0.02
query3	0.26	0.04	0.04
query4	1.78	0.07	0.06
query5	0.54	0.53	0.53
query6	1.23	0.61	0.62
query7	0.02	0.02	0.01
query8	0.04	0.02	0.02
query9	0.52	0.48	0.48
query10	0.54	0.53	0.53
query11	0.11	0.09	0.08
query12	0.12	0.09	0.08
query13	0.62	0.63	0.62
query14	0.79	0.78	0.79
query15	0.77	0.76	0.76
query16	0.37	0.36	0.35
query17	1.00	1.01	1.02
query18	0.21	0.27	0.24
query19	1.88	1.83	1.89
query20	0.02	0.01	0.01
query21	15.44	0.55	0.56
query22	2.05	2.13	2.18
query23	17.36	1.01	0.94
query24	6.30	1.66	0.97
query25	0.37	0.08	0.04
query26	0.76	0.16	0.14
query27	0.04	0.05	0.03
query28	6.22	0.82	0.77
query29	12.63	2.26	2.27
query30	0.58	0.55	0.53
query31	2.80	0.39	0.38
query32	3.38	0.50	0.50
query33	3.09	3.08	3.04
query34	15.25	4.80	4.81
query35	4.89	4.87	4.86
query36	1.07	1.02	1.00
query37	0.06	0.05	0.05
query38	0.03	0.02	0.02
query39	0.02	0.01	0.02
query40	0.15	0.14	0.14
query41	0.07	0.02	0.01
query42	0.02	0.01	0.01
query43	0.03	0.01	0.02
Total cold run time: 103.52 s
Total hot run time: 31.27 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit f1937a16b23224f451bba79d6fab47f416130a9c with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.4 seconds inserted 10000000 Rows, about 467K ops/s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/planner Issues or PRs related to the query planner kind/test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants