Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](resource) Fix MemTableWriter attach resource context to thread context #47556

Merged
merged 3 commits into from
Feb 10, 2025

Conversation

xinyiZzz
Copy link
Contributor

@xinyiZzz xinyiZzz commented Feb 6, 2025

What problem does this PR solve?

Introduced from: #47274
fix:

*** SIGABRT unknown detail explain (@0x1c75) received by PID 7285 (TID 8020 OR 0x7fbc0ad10640) from PID 7285; stack trace: ***
 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_master/doris/be/src/common/signal_handler.h:421
 1# 0x00007FBDAA304520 in /lib/x86_64-linux-gnu/libc.so.6
 2# pthread_kill at ./nptl/pthread_kill.c:89
 3# raise at ../sysdeps/posix/raise.c:27
 4# abort at ./stdlib/abort.c:81
 5# 0x000055E21156252D in /mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be
 6# 0x000055E211554B6A in /mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be
 7# google::LogMessage::SendToLog() in /mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be
 8# google::LogMessage::Flush() in /mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be
 9# google::LogMessageFatal::~LogMessageFatal() in /mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be
10# doris::MemTable::MemTable(long, std::shared_ptr<doris::TabletSchema>, std::vector<doris::SlotDescriptor*, std::allocator<doris::SlotDescriptor*> > const*, doris::TupleDescriptor*, bool, doris::PartialUpdateInfo*) at /home/zcp/repo_center/doris_master/doris/be/src/olap/memtable.cpp:59
11# doris::MemTableWriter::_reset_mem_table() at /home/zcp/repo_center/doris_master/doris/be/src/olap/memtable_writer.cpp:188
12# doris::MemTableWriter::flush_async() in /mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be
13# doris::MemTableMemoryLimiter::_flush_active_memtables(long) at /home/zcp/repo_center/doris_master/doris/be/src/olap/memtable_memory_limiter.cpp:190
14# doris::MemTableMemoryLimiter::handle_memtable_flush() at /home/zcp/repo_center/doris_master/doris/be/src/olap/memtable_memory_limiter.cpp:144
15# doris::LoadChannelMgr::add_batch(doris::PTabletWriterAddBlockRequest const&, doris::PTabletWriterAddBlockResult*) at /home/zcp/repo_center/doris_master/doris/be/src/runtime/load_channel_mgr.cpp:154
16# std::_Function_handler<void (), doris::PInternalService::tablet_writer_add_block(google::protobuf::RpcController*, doris::PTabletWriterAddBlockRequest const*, doris::PTabletWriterAddBlockResult*, google::protobuf::Closure*)::$_0>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
17# doris::WorkThreadPool<false>::work_thread(int) at /home/zcp/repo_center/doris_master/doris/be/src/util/work_thread_pool.hpp:159
18# execute_native_thread_routine at ../../../../../libstdc++-v3/src/c++11/thread.cc:84
19# start_thread at ./nptl/pthread_create.c:442
20# 0x00007FBDAA3E8850 at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:83

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@xinyiZzz
Copy link
Contributor Author

xinyiZzz commented Feb 6, 2025

run buildall

@Thearas
Copy link
Contributor

Thearas commented Feb 6, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@xinyiZzz xinyiZzz force-pushed the 20250205_fix_resource_ctx2 branch from da64e77 to 40ca00d Compare February 6, 2025 08:41
@xinyiZzz
Copy link
Contributor Author

xinyiZzz commented Feb 6, 2025

run buildall

Copy link
Contributor

@kaijchen kaijchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

github-actions bot commented Feb 6, 2025

PR approved by anyone and no changes requested.

@xinyiZzz
Copy link
Contributor Author

xinyiZzz commented Feb 6, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 31718 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 40ca00dabdecc749c7d7834dd396fb7563af4e44, data reload: false

------ Round 1 ----------------------------------
q1	17608	5214	5407	5214
q2	2046	287	158	158
q3	10431	1257	734	734
q4	10198	1016	523	523
q5	7529	2388	2355	2355
q6	184	168	131	131
q7	895	748	604	604
q8	9357	1389	1124	1124
q9	4945	4745	4895	4745
q10	6838	2307	1890	1890
q11	505	276	246	246
q12	346	348	212	212
q13	17772	3658	3111	3111
q14	238	223	209	209
q15	516	470	452	452
q16	623	623	582	582
q17	582	844	335	335
q18	7054	6178	6226	6178
q19	1216	959	528	528
q20	301	329	187	187
q21	2818	2110	1903	1903
q22	351	340	297	297
Total cold run time: 102353 ms
Total hot run time: 31718 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5137	5112	5088	5088
q2	244	327	231	231
q3	2186	2704	2302	2302
q4	1454	1800	1374	1374
q5	4209	4114	4149	4114
q6	210	164	122	122
q7	1865	1807	1680	1680
q8	2604	2608	2566	2566
q9	7166	7154	7129	7129
q10	3047	3225	2829	2829
q11	593	507	503	503
q12	709	789	661	661
q13	3433	3971	3319	3319
q14	303	293	276	276
q15	509	469	454	454
q16	647	682	639	639
q17	1138	1644	1312	1312
q18	7547	7355	7177	7177
q19	799	831	909	831
q20	2011	2012	1823	1823
q21	5527	5101	4785	4785
q22	620	569	555	555
Total cold run time: 51958 ms
Total hot run time: 49770 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 189576 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 40ca00dabdecc749c7d7834dd396fb7563af4e44, data reload: false

query1	1334	943	951	943
query2	6134	1898	1918	1898
query3	11092	4598	4626	4598
query4	25847	23736	22921	22921
query5	4567	674	496	496
query6	305	209	200	200
query7	3992	525	312	312
query8	313	238	234	234
query9	8477	2489	2489	2489
query10	456	306	251	251
query11	15207	15106	14920	14920
query12	161	103	103	103
query13	1553	514	397	397
query14	8871	6238	6313	6238
query15	213	193	182	182
query16	7231	632	486	486
query17	1162	691	570	570
query18	1968	421	314	314
query19	202	201	171	171
query20	134	121	126	121
query21	209	124	108	108
query22	4572	4520	4312	4312
query23	34363	33460	33471	33460
query24	8752	2472	2483	2472
query25	543	495	428	428
query26	1183	294	154	154
query27	2821	506	332	332
query28	4997	2403	2419	2403
query29	718	554	426	426
query30	217	187	164	164
query31	925	875	788	788
query32	76	65	68	65
query33	552	373	303	303
query34	798	887	510	510
query35	802	872	748	748
query36	1008	1028	907	907
query37	134	149	78	78
query38	4491	4367	4346	4346
query39	1499	1455	1441	1441
query40	218	118	119	118
query41	52	53	51	51
query42	124	106	106	106
query43	511	547	492	492
query44	1328	831	809	809
query45	182	179	171	171
query46	893	1063	648	648
query47	1820	1870	1787	1787
query48	384	421	314	314
query49	823	540	405	405
query50	708	766	398	398
query51	4328	4251	4288	4251
query52	109	101	97	97
query53	237	269	192	192
query54	492	492	425	425
query55	83	81	80	80
query56	263	287	260	260
query57	1174	1153	1148	1148
query58	256	246	248	246
query59	2725	2802	2773	2773
query60	283	280	262	262
query61	118	125	149	125
query62	773	730	665	665
query63	223	195	196	195
query64	4087	1040	682	682
query65	3314	3208	3323	3208
query66	1233	399	319	319
query67	16154	15523	15431	15431
query68	8261	763	500	500
query69	460	292	265	265
query70	1221	1177	1125	1125
query71	416	296	266	266
query72	5697	3540	4043	3540
query73	754	744	357	357
query74	9162	9157	8644	8644
query75	3282	3163	2715	2715
query76	3342	1164	747	747
query77	569	365	281	281
query78	10095	9994	9303	9303
query79	2108	831	579	579
query80	616	541	430	430
query81	489	280	233	233
query82	448	147	119	119
query83	162	176	154	154
query84	236	98	72	72
query85	770	422	300	300
query86	329	293	301	293
query87	4437	4461	4272	4272
query88	3713	2199	2196	2196
query89	399	323	282	282
query90	1904	197	191	191
query91	136	138	107	107
query92	71	64	58	58
query93	1562	1014	566	566
query94	698	417	295	295
query95	352	267	260	260
query96	477	559	267	267
query97	2806	2826	2718	2718
query98	229	204	205	204
query99	1562	1391	1290	1290
Total cold run time: 277537 ms
Total hot run time: 189576 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.77 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 40ca00dabdecc749c7d7834dd396fb7563af4e44, data reload: false

query1	0.04	0.05	0.03
query2	0.07	0.03	0.03
query3	0.24	0.07	0.06
query4	1.60	0.10	0.10
query5	0.42	0.41	0.39
query6	1.17	0.65	0.66
query7	0.03	0.02	0.02
query8	0.04	0.03	0.03
query9	0.61	0.51	0.51
query10	0.57	0.59	0.58
query11	0.15	0.11	0.11
query12	0.14	0.11	0.12
query13	0.61	0.59	0.61
query14	2.69	2.80	2.78
query15	0.92	0.85	0.85
query16	0.38	0.38	0.38
query17	1.04	1.03	1.00
query18	0.22	0.19	0.19
query19	1.86	1.82	2.07
query20	0.01	0.01	0.01
query21	15.38	0.89	0.54
query22	0.75	1.17	0.72
query23	14.88	1.35	0.63
query24	7.69	1.13	0.72
query25	0.56	0.16	0.22
query26	0.69	0.16	0.14
query27	0.04	0.05	0.05
query28	9.30	0.86	0.43
query29	12.57	3.99	3.34
query30	0.26	0.10	0.06
query31	2.81	0.62	0.39
query32	3.23	0.55	0.46
query33	2.97	3.02	3.04
query34	15.84	5.26	4.53
query35	4.56	4.58	4.54
query36	0.66	0.49	0.50
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.04	0.02	0.03
query40	0.17	0.13	0.15
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.02
Total cold run time: 105.5 s
Total hot run time: 30.77 s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 42.09% (11020/26184)
Line Coverage: 32.36% (93040/287503)
Region Coverage: 31.52% (47708/151342)
Branch Coverage: 27.54% (24136/87654)
Coverage Report: http://coverage.selectdb-in.cc/coverage/40ca00dabdecc749c7d7834dd396fb7563af4e44_40ca00dabdecc749c7d7834dd396fb7563af4e44/report/index.html

@xinyiZzz xinyiZzz force-pushed the 20250205_fix_resource_ctx2 branch from 40ca00d to cd7d9e9 Compare February 8, 2025 05:43
@xinyiZzz
Copy link
Contributor Author

xinyiZzz commented Feb 8, 2025

run buildall

yiguolei
yiguolei previously approved these changes Feb 8, 2025
Copy link
Contributor

github-actions bot commented Feb 8, 2025

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Feb 8, 2025
@doris-robot
Copy link

TPC-H: Total hot run time: 32297 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit cd7d9e9b6aa90c67804b485e84458a9e2d5bf906, data reload: false

------ Round 1 ----------------------------------
q1	17578	5291	5250	5250
q2	2052	312	182	182
q3	10526	1282	751	751
q4	10218	1029	553	553
q5	7529	2425	2397	2397
q6	198	169	136	136
q7	927	759	605	605
q8	9314	1402	1145	1145
q9	4895	4906	4804	4804
q10	6836	2315	1918	1918
q11	466	284	243	243
q12	352	354	216	216
q13	17775	3687	3156	3156
q14	240	230	210	210
q15	521	470	457	457
q16	632	607	576	576
q17	610	892	348	348
q18	6682	6208	6203	6203
q19	1833	964	574	574
q20	312	327	190	190
q21	3210	2422	2079	2079
q22	370	340	304	304
Total cold run time: 103076 ms
Total hot run time: 32297 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5285	5231	5231	5231
q2	245	327	225	225
q3	2193	2740	2313	2313
q4	1509	1933	1433	1433
q5	4301	4190	4213	4190
q6	216	163	125	125
q7	1866	1893	1751	1751
q8	2639	2674	2610	2610
q9	7292	7133	7110	7110
q10	3048	3263	2822	2822
q11	585	518	491	491
q12	722	814	632	632
q13	3469	3951	3364	3364
q14	276	295	263	263
q15	528	463	444	444
q16	630	690	648	648
q17	1162	1616	1381	1381
q18	7570	7540	7276	7276
q19	869	893	1015	893
q20	1977	2052	1878	1878
q21	5578	5083	4897	4897
q22	650	587	564	564
Total cold run time: 52610 ms
Total hot run time: 50541 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190323 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit cd7d9e9b6aa90c67804b485e84458a9e2d5bf906, data reload: false

query1	1333	954	963	954
query2	6267	1864	1863	1863
query3	10961	4536	4397	4397
query4	54225	25348	23399	23399
query5	4975	562	470	470
query6	313	191	166	166
query7	4869	495	292	292
query8	310	230	217	217
query9	5495	2476	2482	2476
query10	445	296	251	251
query11	15055	15080	15054	15054
query12	151	106	105	105
query13	1039	510	391	391
query14	10467	6870	6478	6478
query15	201	220	188	188
query16	7282	688	518	518
query17	1111	735	581	581
query18	1545	430	310	310
query19	186	198	173	173
query20	122	120	113	113
query21	204	138	104	104
query22	4376	4419	4360	4360
query23	34089	33373	33276	33276
query24	5642	2421	2428	2421
query25	460	452	397	397
query26	708	235	153	153
query27	1926	511	327	327
query28	2705	2435	2393	2393
query29	566	506	408	408
query30	209	192	158	158
query31	907	884	834	834
query32	70	74	60	60
query33	427	357	305	305
query34	761	863	498	498
query35	820	841	773	773
query36	942	1012	917	917
query37	128	101	103	101
query38	4279	4401	4240	4240
query39	1481	1468	1441	1441
query40	202	120	104	104
query41	53	49	48	48
query42	117	107	114	107
query43	511	519	474	474
query44	1300	804	789	789
query45	181	170	170	170
query46	877	1068	670	670
query47	1811	1892	1806	1806
query48	390	419	304	304
query49	692	498	415	415
query50	707	747	409	409
query51	4291	4308	4258	4258
query52	106	106	97	97
query53	242	253	198	198
query54	475	507	426	426
query55	82	85	90	85
query56	248	319	282	282
query57	1207	1203	1123	1123
query58	268	252	254	252
query59	2911	2935	2928	2928
query60	306	310	263	263
query61	122	117	123	117
query62	732	725	667	667
query63	234	191	188	188
query64	1783	1028	648	648
query65	3191	3154	3121	3121
query66	728	413	298	298
query67	16086	15520	15449	15449
query68	4661	778	517	517
query69	500	312	261	261
query70	1209	1133	1158	1133
query71	423	302	265	265
query72	6140	3710	3786	3710
query73	1097	754	337	337
query74	8892	9329	8724	8724
query75	3185	3158	2721	2721
query76	3709	1165	725	725
query77	532	368	261	261
query78	10147	10087	9346	9346
query79	1841	815	585	585
query80	665	629	444	444
query81	503	280	231	231
query82	287	151	127	127
query83	181	166	151	151
query84	297	91	75	75
query85	743	341	297	297
query86	328	302	289	289
query87	4434	4521	4409	4409
query88	2785	2154	2140	2140
query89	390	326	290	290
query90	1762	188	184	184
query91	134	182	107	107
query92	64	59	55	55
query93	2343	1002	574	574
query94	674	379	309	309
query95	338	258	246	246
query96	487	553	267	267
query97	2780	2899	2776	2776
query98	226	204	200	200
query99	1282	1418	1309	1309
Total cold run time: 289944 ms
Total hot run time: 190323 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.72 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit cd7d9e9b6aa90c67804b485e84458a9e2d5bf906, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.03
query3	0.23	0.07	0.06
query4	1.62	0.10	0.11
query5	0.41	0.42	0.42
query6	1.16	0.68	0.66
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.60	0.52	0.52
query10	0.60	0.58	0.57
query11	0.15	0.10	0.10
query12	0.14	0.11	0.12
query13	0.62	0.60	0.59
query14	2.67	2.75	2.72
query15	0.94	0.85	0.84
query16	0.39	0.37	0.39
query17	1.02	1.05	1.04
query18	0.22	0.20	0.19
query19	1.93	1.82	2.03
query20	0.01	0.01	0.01
query21	15.42	0.90	0.53
query22	0.75	1.26	0.88
query23	14.76	1.38	0.64
query24	6.73	1.69	0.68
query25	0.50	0.24	0.15
query26	0.59	0.16	0.15
query27	0.05	0.05	0.06
query28	9.38	0.89	0.43
query29	12.61	4.01	3.32
query30	0.25	0.09	0.06
query31	2.82	0.58	0.38
query32	3.22	0.55	0.47
query33	3.02	3.00	3.08
query34	15.79	5.11	4.47
query35	4.55	4.55	4.50
query36	0.66	0.50	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.03
query40	0.16	0.14	0.13
query41	0.08	0.03	0.02
query42	0.04	0.03	0.02
query43	0.03	0.03	0.02
Total cold run time: 104.46 s
Total hot run time: 30.72 s

@xinyiZzz
Copy link
Contributor Author

xinyiZzz commented Feb 8, 2025

run buildall

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Feb 8, 2025
@doris-robot
Copy link

TPC-H: Total hot run time: 31401 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 82d9608a31b0f53c406f865b56d05fbfa2b5500a, data reload: false

------ Round 1 ----------------------------------
q1	17607	5317	5077	5077
q2	2057	300	176	176
q3	10605	1267	762	762
q4	10203	997	527	527
q5	7519	2436	2324	2324
q6	192	166	133	133
q7	904	749	599	599
q8	9308	1346	1099	1099
q9	4849	4493	4663	4493
q10	6816	2317	1896	1896
q11	485	272	254	254
q12	344	360	220	220
q13	17765	3689	3069	3069
q14	241	234	212	212
q15	500	459	462	459
q16	643	605	603	603
q17	595	880	338	338
q18	6575	6205	6276	6205
q19	1533	964	557	557
q20	313	315	180	180
q21	2775	2137	1922	1922
q22	378	340	296	296
Total cold run time: 102207 ms
Total hot run time: 31401 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5166	5146	5137	5137
q2	240	337	255	255
q3	2184	2745	2313	2313
q4	1441	1842	1380	1380
q5	4236	4185	4169	4169
q6	212	170	125	125
q7	1932	1861	1668	1668
q8	2622	2601	2588	2588
q9	7261	7225	7146	7146
q10	3015	3196	2831	2831
q11	600	536	502	502
q12	692	792	619	619
q13	3360	3833	3306	3306
q14	283	288	291	288
q15	514	462	449	449
q16	657	704	647	647
q17	1142	1658	1306	1306
q18	7610	7423	7274	7274
q19	822	833	829	829
q20	2000	1998	1976	1976
q21	5454	5004	4792	4792
q22	601	590	522	522
Total cold run time: 52044 ms
Total hot run time: 50122 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190635 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 82d9608a31b0f53c406f865b56d05fbfa2b5500a, data reload: false

query1	1296	980	935	935
query2	6212	1817	1858	1817
query3	11035	4459	4732	4459
query4	55107	24676	23315	23315
query5	5249	558	490	490
query6	360	193	185	185
query7	5069	521	300	300
query8	323	245	234	234
query9	6285	2487	2492	2487
query10	428	318	255	255
query11	15290	15202	14828	14828
query12	163	109	111	109
query13	1143	533	412	412
query14	10395	6470	7287	6470
query15	209	210	198	198
query16	7047	674	472	472
query17	1081	749	593	593
query18	1554	420	325	325
query19	212	212	171	171
query20	125	123	131	123
query21	215	125	111	111
query22	4538	4450	4392	4392
query23	34076	33434	33514	33434
query24	5697	2447	2510	2447
query25	489	459	418	418
query26	748	277	160	160
query27	1994	480	334	334
query28	2834	2453	2418	2418
query29	621	586	446	446
query30	216	190	154	154
query31	932	883	840	840
query32	78	67	60	60
query33	469	358	305	305
query34	783	856	511	511
query35	792	855	737	737
query36	975	988	932	932
query37	131	110	78	78
query38	4345	4381	4300	4300
query39	1645	1480	1465	1465
query40	213	117	108	108
query41	51	53	50	50
query42	123	108	111	108
query43	513	533	495	495
query44	1318	804	800	800
query45	187	175	170	170
query46	886	1066	681	681
query47	1849	1867	1794	1794
query48	392	428	309	309
query49	714	525	413	413
query50	729	758	430	430
query51	4263	4292	4181	4181
query52	107	105	97	97
query53	249	277	201	201
query54	515	540	441	441
query55	85	81	83	81
query56	281	289	267	267
query57	1169	1175	1117	1117
query58	251	247	253	247
query59	2745	2981	2676	2676
query60	321	275	268	268
query61	123	122	143	122
query62	752	747	673	673
query63	242	202	201	201
query64	2185	1047	703	703
query65	3369	3273	3176	3176
query66	755	395	301	301
query67	15910	15545	15319	15319
query68	6116	772	515	515
query69	570	302	264	264
query70	1190	1175	1116	1116
query71	445	321	266	266
query72	6101	3723	3797	3723
query73	1120	758	343	343
query74	9298	9004	9012	9004
query75	3269	3179	2691	2691
query76	3956	1181	750	750
query77	555	385	285	285
query78	10015	9969	9232	9232
query79	2396	841	615	615
query80	648	560	451	451
query81	488	287	242	242
query82	621	157	117	117
query83	183	169	154	154
query84	291	93	79	79
query85	774	357	416	357
query86	399	312	279	279
query87	4517	4553	4549	4549
query88	3547	2170	2147	2147
query89	417	313	284	284
query90	1781	195	193	193
query91	144	146	107	107
query92	73	63	55	55
query93	1906	1041	582	582
query94	693	405	320	320
query95	353	272	260	260
query96	487	557	270	270
query97	2791	2852	2742	2742
query98	231	207	209	207
query99	1704	1416	1287	1287
Total cold run time: 297537 ms
Total hot run time: 190635 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.11 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 82d9608a31b0f53c406f865b56d05fbfa2b5500a, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.04	0.03
query3	0.25	0.06	0.07
query4	1.62	0.10	0.10
query5	0.42	0.42	0.39
query6	1.16	0.68	0.65
query7	0.02	0.02	0.01
query8	0.04	0.02	0.04
query9	0.61	0.52	0.53
query10	0.58	0.60	0.58
query11	0.16	0.11	0.11
query12	0.14	0.11	0.11
query13	0.62	0.60	0.59
query14	2.70	2.81	2.70
query15	0.92	0.86	0.85
query16	0.38	0.37	0.37
query17	1.04	1.04	1.03
query18	0.21	0.20	0.20
query19	1.92	1.81	2.01
query20	0.01	0.01	0.01
query21	15.37	0.94	0.55
query22	0.77	1.15	0.98
query23	14.70	1.40	0.65
query24	6.86	1.05	1.21
query25	0.44	0.37	0.07
query26	0.58	0.18	0.15
query27	0.05	0.05	0.05
query28	9.66	0.94	0.44
query29	12.56	4.02	3.35
query30	0.25	0.09	0.07
query31	2.82	0.60	0.40
query32	3.22	0.53	0.46
query33	2.96	2.98	3.00
query34	15.75	5.14	4.47
query35	4.54	4.53	4.48
query36	0.65	0.50	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.04	0.02	0.02
query40	0.18	0.14	0.13
query41	0.08	0.02	0.02
query42	0.04	0.03	0.02
query43	0.04	0.04	0.02
Total cold run time: 104.6 s
Total hot run time: 31.11 s

@xinyiZzz
Copy link
Contributor Author

xinyiZzz commented Feb 9, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 31733 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 15bbc3df5d701a459cf3de721990ad192245346e, data reload: false

------ Round 1 ----------------------------------
q1	17583	5259	5136	5136
q2	2047	311	174	174
q3	10404	1288	763	763
q4	10235	1025	531	531
q5	7622	2476	2357	2357
q6	194	170	137	137
q7	930	791	605	605
q8	9322	1338	1085	1085
q9	4937	4747	4704	4704
q10	6832	2326	1886	1886
q11	481	288	255	255
q12	353	368	232	232
q13	17767	3708	3165	3165
q14	224	231	215	215
q15	507	483	462	462
q16	641	601	593	593
q17	594	890	355	355
q18	6539	6361	6094	6094
q19	1470	959	564	564
q20	312	332	198	198
q21	2807	2152	1914	1914
q22	359	328	308	308
Total cold run time: 102160 ms
Total hot run time: 31733 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5220	5168	5151	5151
q2	239	329	233	233
q3	2171	2712	2347	2347
q4	1477	1841	1385	1385
q5	4241	4161	4164	4161
q6	204	169	123	123
q7	1892	1861	1754	1754
q8	2664	2592	2577	2577
q9	7327	7118	7183	7118
q10	3055	3242	2818	2818
q11	592	514	482	482
q12	665	793	644	644
q13	3382	3963	3356	3356
q14	291	300	277	277
q15	502	488	453	453
q16	643	712	646	646
q17	1152	1631	1362	1362
q18	7761	7388	7236	7236
q19	806	825	957	825
q20	2011	2010	1879	1879
q21	5443	5046	4778	4778
q22	598	588	537	537
Total cold run time: 52336 ms
Total hot run time: 50142 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190681 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 15bbc3df5d701a459cf3de721990ad192245346e, data reload: false

query1	1304	938	945	938
query2	6119	1920	1878	1878
query3	11104	4528	4416	4416
query4	54813	25488	23263	23263
query5	5002	576	471	471
query6	336	201	185	185
query7	4947	526	296	296
query8	310	247	243	243
query9	5771	2636	2625	2625
query10	441	313	253	253
query11	15051	15189	14809	14809
query12	153	109	106	106
query13	1070	520	397	397
query14	10788	6920	6686	6686
query15	204	205	184	184
query16	7137	707	499	499
query17	1082	755	579	579
query18	1592	428	326	326
query19	217	204	173	173
query20	131	130	127	127
query21	213	134	109	109
query22	4494	4439	4301	4301
query23	34123	33550	33508	33508
query24	5658	2427	2440	2427
query25	466	481	423	423
query26	691	286	158	158
query27	1736	496	345	345
query28	2775	2480	2451	2451
query29	576	552	422	422
query30	228	199	156	156
query31	898	870	807	807
query32	77	65	60	60
query33	427	372	303	303
query34	800	866	527	527
query35	791	842	746	746
query36	985	999	932	932
query37	133	102	88	88
query38	4346	4351	4316	4316
query39	1475	1438	1424	1424
query40	216	116	100	100
query41	50	54	48	48
query42	125	106	105	105
query43	522	539	499	499
query44	1352	825	816	816
query45	187	177	164	164
query46	888	1065	678	678
query47	1833	1849	1796	1796
query48	396	438	329	329
query49	704	525	431	431
query50	746	753	424	424
query51	4353	4291	4332	4291
query52	106	106	95	95
query53	239	274	189	189
query54	481	506	414	414
query55	88	86	82	82
query56	287	276	274	274
query57	1174	1170	1135	1135
query58	250	245	266	245
query59	2886	2921	2711	2711
query60	293	266	258	258
query61	119	122	117	117
query62	720	760	670	670
query63	235	201	194	194
query64	1664	1036	676	676
query65	3232	3136	3145	3136
query66	746	393	304	304
query67	15963	15696	15276	15276
query68	5521	787	512	512
query69	525	313	267	267
query70	1231	1166	1168	1166
query71	430	312	267	267
query72	6385	3667	3765	3667
query73	1297	758	351	351
query74	9024	9329	8927	8927
query75	3178	3152	2732	2732
query76	3838	1271	735	735
query77	529	365	283	283
query78	10008	10203	9245	9245
query79	2251	801	591	591
query80	646	531	445	445
query81	506	284	233	233
query82	230	155	119	119
query83	172	168	158	158
query84	288	90	72	72
query85	779	349	306	306
query86	340	312	276	276
query87	4472	4602	4474	4474
query88	3156	2239	2208	2208
query89	382	336	279	279
query90	1740	189	198	189
query91	137	136	107	107
query92	74	59	54	54
query93	2397	997	580	580
query94	690	419	308	308
query95	340	272	262	262
query96	487	571	272	272
query97	2789	2899	2685	2685
query98	242	209	196	196
query99	1296	1386	1301	1301
Total cold run time: 293699 ms
Total hot run time: 190681 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.64 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 15bbc3df5d701a459cf3de721990ad192245346e, data reload: false

query1	0.04	0.05	0.04
query2	0.07	0.04	0.04
query3	0.25	0.06	0.06
query4	1.63	0.10	0.10
query5	0.42	0.44	0.40
query6	1.15	0.64	0.67
query7	0.03	0.02	0.01
query8	0.04	0.03	0.03
query9	0.61	0.53	0.51
query10	0.56	0.59	0.58
query11	0.15	0.10	0.10
query12	0.15	0.12	0.11
query13	0.61	0.59	0.60
query14	2.64	2.82	2.81
query15	0.92	0.84	0.84
query16	0.38	0.38	0.37
query17	1.04	1.05	1.03
query18	0.22	0.20	0.20
query19	1.88	1.81	1.99
query20	0.01	0.01	0.02
query21	15.36	0.91	0.54
query22	0.73	1.13	0.68
query23	14.96	1.38	0.67
query24	6.61	1.13	0.63
query25	0.52	0.24	0.13
query26	0.58	0.16	0.13
query27	0.05	0.05	0.06
query28	9.54	0.93	0.43
query29	12.55	3.90	3.29
query30	0.24	0.08	0.06
query31	2.82	0.60	0.38
query32	3.26	0.57	0.46
query33	2.98	2.98	3.02
query34	15.94	5.17	4.59
query35	4.57	4.55	4.54
query36	0.66	0.48	0.50
query37	0.10	0.06	0.06
query38	0.05	0.04	0.04
query39	0.02	0.03	0.03
query40	0.16	0.12	0.12
query41	0.09	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 104.65 s
Total hot run time: 30.64 s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 42.62% (11169/26209)
Line Coverage: 32.65% (93960/287807)
Region Coverage: 31.80% (48191/151539)
Branch Coverage: 27.71% (24332/87794)
Coverage Report: http://coverage.selectdb-in.cc/coverage/15bbc3df5d701a459cf3de721990ad192245346e_15bbc3df5d701a459cf3de721990ad192245346e/report/index.html

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Feb 10, 2025
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@xinyiZzz xinyiZzz merged commit 16fbce8 into apache:master Feb 10, 2025
23 of 24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants