Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feat](Nereids) support copy into command #47194

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

LiBinfeng-01
Copy link
Collaborator

@LiBinfeng-01 LiBinfeng-01 commented Jan 18, 2025

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

support copy into command in Nereids as Original Planner

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Jan 18, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@LiBinfeng-01
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32488 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 04a4daace71726ea8e25e516364115944f00d901, data reload: false

------ Round 1 ----------------------------------
q1	17587	5522	5403	5403
q2	2052	305	198	198
q3	10397	1236	740	740
q4	10238	963	534	534
q5	8130	2357	2175	2175
q6	191	165	136	136
q7	901	776	615	615
q8	9247	1376	1180	1180
q9	5219	4871	4866	4866
q10	6893	2321	1923	1923
q11	498	276	255	255
q12	358	366	236	236
q13	17782	3681	3119	3119
q14	230	229	207	207
q15	541	487	477	477
q16	619	619	583	583
q17	563	865	325	325
q18	6847	6404	6505	6404
q19	3269	943	568	568
q20	307	329	186	186
q21	2895	2267	2039	2039
q22	369	346	319	319
Total cold run time: 105133 ms
Total hot run time: 32488 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5654	5522	5568	5522
q2	247	328	236	236
q3	2301	2652	2370	2370
q4	1480	1831	1399	1399
q5	4346	4796	4780	4780
q6	172	160	133	133
q7	2075	1971	1850	1850
q8	2660	2850	2738	2738
q9	7309	7202	7173	7173
q10	2980	3307	2761	2761
q11	568	513	478	478
q12	660	765	663	663
q13	3623	3938	3321	3321
q14	301	312	268	268
q15	534	495	475	475
q16	656	667	649	649
q17	1273	1762	1279	1279
q18	7823	7425	7460	7425
q19	815	1185	1035	1035
q20	2035	2074	1929	1929
q21	5787	5251	5063	5063
q22	652	644	599	599
Total cold run time: 53951 ms
Total hot run time: 52146 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 194582 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 04a4daace71726ea8e25e516364115944f00d901, data reload: false

query1	1334	991	929	929
query2	6356	2034	1960	1960
query3	11002	4479	4482	4479
query4	60628	29720	23277	23277
query5	5413	595	456	456
query6	435	205	185	185
query7	5573	498	293	293
query8	338	233	244	233
query9	8289	2570	2577	2570
query10	443	296	253	253
query11	17289	15156	15660	15156
query12	165	110	120	110
query13	1420	535	421	421
query14	10368	7942	7246	7246
query15	211	209	189	189
query16	7361	645	496	496
query17	1132	727	588	588
query18	1861	434	333	333
query19	202	186	164	164
query20	114	123	117	117
query21	217	121	109	109
query22	4583	4838	4604	4604
query23	34163	33133	33481	33133
query24	5501	2322	2295	2295
query25	467	500	418	418
query26	639	279	152	152
query27	1651	463	337	337
query28	4185	2475	2418	2418
query29	534	565	431	431
query30	216	188	160	160
query31	883	880	830	830
query32	72	61	57	57
query33	432	348	293	293
query34	733	852	527	527
query35	855	823	789	789
query36	994	1041	950	950
query37	125	106	83	83
query38	4479	4260	4383	4260
query39	1499	1439	1472	1439
query40	220	113	99	99
query41	52	48	48	48
query42	117	103	103	103
query43	513	536	490	490
query44	1330	815	816	815
query45	185	181	180	180
query46	917	1073	653	653
query47	1871	1879	1852	1852
query48	403	416	328	328
query49	715	506	422	422
query50	659	663	401	401
query51	6980	7057	7009	7009
query52	103	95	92	92
query53	227	258	187	187
query54	499	511	422	422
query55	88	87	85	85
query56	263	270	266	266
query57	1171	1155	1054	1054
query58	234	241	228	228
query59	2888	3058	2919	2919
query60	278	269	241	241
query61	116	117	119	117
query62	691	719	648	648
query63	223	186	185	185
query64	1237	1028	671	671
query65	3236	3214	3143	3143
query66	646	400	303	303
query67	16028	15532	15508	15508
query68	5015	845	529	529
query69	485	299	265	265
query70	1198	1080	1083	1080
query71	435	285	260	260
query72	6372	3898	3825	3825
query73	818	737	349	349
query74	10210	9066	8935	8935
query75	3305	3158	2695	2695
query76	3772	1164	759	759
query77	470	361	272	272
query78	10151	10108	9398	9398
query79	2612	794	602	602
query80	1707	525	444	444
query81	558	272	237	237
query82	350	145	134	134
query83	284	170	147	147
query84	284	100	74	74
query85	787	420	314	314
query86	425	313	310	310
query87	4527	4567	4460	4460
query88	3780	2155	2109	2109
query89	397	319	284	284
query90	1599	190	192	190
query91	139	135	111	111
query92	64	59	52	52
query93	2626	856	525	525
query94	749	409	302	302
query95	334	263	266	263
query96	483	607	275	275
query97	2815	2850	2718	2718
query98	221	209	200	200
query99	1282	1429	1255	1255
Total cold run time: 312306 ms
Total hot run time: 194582 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.44 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 04a4daace71726ea8e25e516364115944f00d901, data reload: false

query1	0.03	0.03	0.04
query2	0.07	0.03	0.03
query3	0.25	0.06	0.06
query4	1.63	0.10	0.11
query5	0.41	0.41	0.42
query6	1.16	0.64	0.66
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.59	0.52	0.50
query10	0.56	0.55	0.56
query11	0.14	0.11	0.10
query12	0.14	0.11	0.11
query13	0.60	0.59	0.61
query14	2.68	2.71	2.73
query15	0.90	0.83	0.82
query16	0.40	0.38	0.40
query17	1.00	1.05	1.06
query18	0.23	0.21	0.21
query19	1.96	1.80	1.98
query20	0.01	0.01	0.01
query21	15.36	0.98	0.61
query22	0.76	0.90	0.85
query23	14.97	1.39	0.56
query24	2.77	1.52	1.45
query25	0.13	0.19	0.23
query26	0.22	0.14	0.13
query27	0.05	0.05	0.06
query28	14.75	1.00	0.43
query29	13.79	3.89	3.25
query30	0.25	0.08	0.06
query31	2.82	0.60	0.40
query32	3.22	0.55	0.46
query33	2.98	3.05	3.02
query34	16.59	5.15	4.49
query35	4.54	4.54	4.54
query36	0.61	0.49	0.48
query37	0.09	0.06	0.06
query38	0.04	0.04	0.03
query39	0.03	0.03	0.03
query40	0.16	0.14	0.13
query41	0.08	0.03	0.02
query42	0.04	0.03	0.02
query43	0.04	0.03	0.04
Total cold run time: 107.11 s
Total hot run time: 31.44 s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants