Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

branch-3.0: [improve](cloud-mow)Add delete bitmap metrics #47028 #47533

Merged
merged 1 commit into from
Feb 6, 2025

Conversation

github-actions[bot]
Copy link
Contributor

@github-actions github-actions bot commented Feb 6, 2025

Cherry-picked from #47028

Count the max delete bitmap of tablet and base rowset, adn put them to
be metrics.
@github-actions github-actions bot requested a review from dataroaring as a code owner February 6, 2025 02:27
@Thearas
Copy link
Contributor

Thearas commented Feb 6, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring closed this Feb 6, 2025
@dataroaring dataroaring reopened this Feb 6, 2025
@Thearas
Copy link
Contributor

Thearas commented Feb 6, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41016 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit eb91b0e9caa04d06905f97815033d941a241dc7a, data reload: false

------ Round 1 ----------------------------------
q1	17566	7444	7270	7270
q2	2075	174	171	171
q3	10609	1089	1222	1089
q4	10257	726	674	674
q5	7718	2914	2813	2813
q6	241	150	147	147
q7	978	625	612	612
q8	9347	1960	2037	1960
q9	6617	6423	6399	6399
q10	7049	2319	2330	2319
q11	466	260	307	260
q12	414	203	210	203
q13	17786	2994	3036	2994
q14	230	213	211	211
q15	588	530	536	530
q16	683	587	589	587
q17	989	591	610	591
q18	7316	6659	6668	6659
q19	1413	1062	1057	1057
q20	480	209	201	201
q21	4340	3284	3265	3265
q22	1136	1014	1004	1004
Total cold run time: 108298 ms
Total hot run time: 41016 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7255	7255	7270	7255
q2	331	230	228	228
q3	2948	2901	2886	2886
q4	2042	1881	1850	1850
q5	5767	5731	5753	5731
q6	226	144	143	143
q7	2269	1843	1817	1817
q8	3338	3658	3524	3524
q9	8727	8912	8839	8839
q10	3577	3542	3543	3542
q11	609	532	504	504
q12	811	640	571	571
q13	9753	3189	3175	3175
q14	297	275	258	258
q15	576	519	525	519
q16	704	660	652	652
q17	1850	1646	1595	1595
q18	8384	7716	7665	7665
q19	1670	1541	1465	1465
q20	2125	1892	1885	1885
q21	5414	5289	5348	5289
q22	1151	1071	1058	1058
Total cold run time: 69824 ms
Total hot run time: 60451 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 198493 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit eb91b0e9caa04d06905f97815033d941a241dc7a, data reload: false

query1	1273	914	885	885
query2	6248	2081	2063	2063
query3	10803	4308	4327	4308
query4	66373	28895	23585	23585
query5	5110	462	474	462
query6	430	197	185	185
query7	5684	311	315	311
query8	308	225	223	223
query9	9330	2713	2691	2691
query10	472	261	259	259
query11	18029	15650	15972	15650
query12	156	106	105	105
query13	1569	437	415	415
query14	10731	7719	7255	7255
query15	203	182	191	182
query16	7127	504	489	489
query17	1105	601	619	601
query18	1842	325	338	325
query19	227	160	166	160
query20	116	110	110	110
query21	226	105	107	105
query22	4712	4408	4338	4338
query23	34733	34116	34365	34116
query24	6163	2939	2972	2939
query25	543	424	411	411
query26	651	166	167	166
query27	1841	346	360	346
query28	4454	2501	2443	2443
query29	700	461	432	432
query30	244	164	160	160
query31	998	819	834	819
query32	70	60	59	59
query33	471	293	287	287
query34	904	498	541	498
query35	834	742	718	718
query36	1091	979	979	979
query37	126	76	74	74
query38	4191	4016	4014	4014
query39	1512	1480	1449	1449
query40	213	99	100	99
query41	50	48	48	48
query42	119	97	98	97
query43	535	491	498	491
query44	1198	853	849	849
query45	188	168	166	166
query46	1148	730	722	722
query47	2083	1911	1941	1911
query48	481	397	399	397
query49	746	391	390	390
query50	841	437	425	425
query51	7373	7207	7163	7163
query52	104	86	90	86
query53	254	185	183	183
query54	570	444	445	444
query55	80	82	74	74
query56	264	244	250	244
query57	1231	1114	1077	1077
query58	205	206	203	203
query59	3055	2951	2979	2951
query60	273	254	256	254
query61	108	168	105	105
query62	843	743	731	731
query63	216	190	185	185
query64	1412	690	664	664
query65	3308	3237	3200	3200
query66	709	287	301	287
query67	16287	15639	15460	15460
query68	3836	600	590	590
query69	420	275	263	263
query70	1160	1123	1123	1123
query71	355	263	261	261
query72	6372	4190	4094	4094
query73	749	340	351	340
query74	10085	9000	9174	9000
query75	3351	2691	2673	2673
query76	1900	1108	1102	1102
query77	508	275	303	275
query78	10541	9574	9720	9574
query79	1442	602	609	602
query80	862	433	418	418
query81	505	243	239	239
query82	1174	121	122	121
query83	156	146	146	146
query84	287	84	76	76
query85	834	303	289	289
query86	323	294	275	275
query87	4410	4295	4318	4295
query88	3648	2393	2336	2336
query89	411	309	300	300
query90	2018	191	190	190
query91	187	148	148	148
query92	70	50	55	50
query93	1758	561	567	561
query94	771	311	273	273
query95	363	263	257	257
query96	609	276	282	276
query97	3431	3248	3194	3194
query98	212	202	204	202
query99	1646	1454	1427	1427
Total cold run time: 319888 ms
Total hot run time: 198493 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.57 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit eb91b0e9caa04d06905f97815033d941a241dc7a, data reload: false

query1	0.03	0.03	0.03
query2	0.08	0.02	0.03
query3	0.23	0.06	0.06
query4	1.62	0.10	0.10
query5	0.53	0.50	0.51
query6	1.14	0.73	0.73
query7	0.02	0.02	0.02
query8	0.04	0.04	0.03
query9	0.56	0.52	0.50
query10	0.55	0.54	0.57
query11	0.14	0.11	0.11
query12	0.15	0.12	0.11
query13	0.61	0.59	0.58
query14	2.73	2.78	2.86
query15	0.92	0.84	0.85
query16	0.37	0.38	0.38
query17	1.06	1.07	1.06
query18	0.23	0.21	0.22
query19	2.00	1.84	2.02
query20	0.01	0.01	0.01
query21	15.45	0.59	0.59
query22	2.54	2.68	1.62
query23	16.98	0.87	0.88
query24	3.15	0.76	1.27
query25	0.18	0.13	0.08
query26	0.53	0.13	0.13
query27	0.04	0.05	0.05
query28	10.65	1.10	1.07
query29	12.57	3.27	3.25
query30	0.24	0.06	0.06
query31	2.85	0.39	0.39
query32	3.25	0.47	0.45
query33	2.99	3.03	3.04
query34	17.08	4.50	4.54
query35	4.52	4.53	4.48
query36	0.67	0.49	0.49
query37	0.09	0.06	0.07
query38	0.04	0.03	0.04
query39	0.03	0.03	0.02
query40	0.16	0.13	0.12
query41	0.07	0.03	0.02
query42	0.04	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 107.17 s
Total hot run time: 32.57 s

@dataroaring dataroaring merged commit 91eb0be into branch-3.0 Feb 6, 2025
21 of 22 checks passed
@github-actions github-actions bot deleted the auto-pick-47028-branch-3.0 branch February 6, 2025 10:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants