Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

branch-3.0-pick: [Fix](partial update) Fix incorrect result when partial update include delete sign columns #46194 #46336

Open
wants to merge 1 commit into
base: branch-3.0
Choose a base branch
from

Conversation

bobhan1
Copy link
Contributor

@bobhan1 bobhan1 commented Jan 3, 2025

pick #46194

…e delete sign columns (apache#46194)

Currently, when a partial update load include delete sign columns, it
will not read delete sign columns from historical data. This may result
in incorrect result because it may read data from rows whose delete sign
is 1.
@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@bobhan1
Copy link
Contributor Author

bobhan1 commented Jan 3, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40791 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e9afb8d2fd75de54e261b9550c8649db554472ea, data reload: false

------ Round 1 ----------------------------------
q1	17574	7532	7232	7232
q2	2062	180	166	166
q3	10769	1133	1246	1133
q4	10577	740	754	740
q5	7740	2863	2830	2830
q6	240	150	143	143
q7	973	609	610	609
q8	9370	1952	2075	1952
q9	6628	6410	6416	6410
q10	7032	2335	2329	2329
q11	473	256	270	256
q12	403	210	207	207
q13	17775	2960	3055	2960
q14	237	204	209	204
q15	560	530	531	530
q16	681	617	598	598
q17	991	638	580	580
q18	7314	6608	6704	6608
q19	1396	1016	1031	1016
q20	487	207	197	197
q21	4034	3206	3112	3112
q22	1099	979	997	979
Total cold run time: 108415 ms
Total hot run time: 40791 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7222	7200	7185	7185
q2	319	242	233	233
q3	2919	2914	2964	2914
q4	2027	1838	1819	1819
q5	5739	5772	5775	5772
q6	221	138	140	138
q7	2261	1864	1844	1844
q8	3364	3634	3506	3506
q9	8875	8886	8832	8832
q10	3609	3576	3593	3576
q11	598	489	490	489
q12	825	610	651	610
q13	10321	3127	3216	3127
q14	297	271	268	268
q15	572	533	522	522
q16	725	662	683	662
q17	1867	1632	1618	1618
q18	8264	7827	7515	7515
q19	1664	1615	1655	1615
q20	2114	1894	1887	1887
q21	5474	5398	5366	5366
q22	1127	1022	1062	1022
Total cold run time: 70404 ms
Total hot run time: 60520 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197499 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e9afb8d2fd75de54e261b9550c8649db554472ea, data reload: false

query1	1295	956	938	938
query2	6219	2062	2030	2030
query3	10793	4283	4247	4247
query4	66231	28862	23557	23557
query5	5037	463	454	454
query6	410	172	174	172
query7	5564	324	308	308
query8	326	232	230	230
query9	8750	2710	2716	2710
query10	449	269	252	252
query11	17368	15237	15713	15237
query12	161	104	101	101
query13	1526	459	426	426
query14	10853	7157	7113	7113
query15	204	177	186	177
query16	7204	494	476	476
query17	1105	589	590	589
query18	1906	360	325	325
query19	222	157	161	157
query20	122	113	114	113
query21	64	47	46	46
query22	4694	4799	4559	4559
query23	34518	34397	34124	34124
query24	6046	2907	2892	2892
query25	533	430	422	422
query26	652	162	172	162
query27	1699	299	307	299
query28	4253	2513	2480	2480
query29	730	447	417	417
query30	244	171	173	171
query31	1017	803	825	803
query32	72	53	57	53
query33	430	283	275	275
query34	920	502	520	502
query35	874	745	758	745
query36	1082	954	917	917
query37	124	71	77	71
query38	4117	4077	4000	4000
query39	1511	1473	1469	1469
query40	142	81	79	79
query41	49	46	46	46
query42	109	95	96	95
query43	549	496	482	482
query44	1192	826	842	826
query45	187	171	170	170
query46	1171	737	726	726
query47	2044	1960	1949	1949
query48	464	382	380	380
query49	736	403	382	382
query50	859	407	414	407
query51	7362	7216	7240	7216
query52	93	86	84	84
query53	250	179	180	179
query54	554	469	443	443
query55	76	74	72	72
query56	257	240	231	231
query57	1223	1160	1113	1113
query58	211	208	202	202
query59	3267	3045	3060	3045
query60	265	259	248	248
query61	106	110	114	110
query62	768	647	670	647
query63	206	194	193	193
query64	1396	671	657	657
query65	3244	3204	3180	3180
query66	635	302	303	302
query67	15827	15666	15609	15609
query68	4149	563	578	563
query69	409	263	255	255
query70	1213	1024	1055	1024
query71	325	254	265	254
query72	6514	3982	4047	3982
query73	753	345	332	332
query74	9408	8952	9098	8952
query75	3362	2653	2639	2639
query76	1838	1101	1030	1030
query77	534	269	274	269
query78	10692	9571	9603	9571
query79	1103	593	597	593
query80	829	426	428	426
query81	506	244	240	240
query82	1273	117	110	110
query83	234	139	146	139
query84	287	77	74	74
query85	889	310	287	287
query86	334	289	293	289
query87	4458	4328	4325	4325
query88	3633	2401	2421	2401
query89	413	295	286	286
query90	1877	180	180	180
query91	181	144	152	144
query92	70	47	58	47
query93	1327	538	543	538
query94	740	285	293	285
query95	346	252	248	248
query96	610	275	271	271
query97	3338	3215	3167	3167
query98	216	197	200	197
query99	1576	1298	1289	1289
Total cold run time: 316004 ms
Total hot run time: 197499 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.38 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e9afb8d2fd75de54e261b9550c8649db554472ea, data reload: false

query1	0.03	0.03	0.04
query2	0.07	0.03	0.03
query3	0.24	0.06	0.06
query4	1.63	0.11	0.10
query5	0.54	0.53	0.51
query6	1.13	0.74	0.72
query7	0.02	0.02	0.01
query8	0.04	0.03	0.06
query9	0.57	0.50	0.49
query10	0.55	0.55	0.55
query11	0.15	0.11	0.10
query12	0.13	0.12	0.10
query13	0.60	0.59	0.60
query14	2.92	3.02	3.03
query15	0.89	0.84	0.82
query16	0.39	0.39	0.38
query17	1.05	1.06	1.08
query18	0.23	0.22	0.23
query19	1.94	2.00	1.80
query20	0.01	0.01	0.01
query21	15.36	0.60	0.58
query22	2.95	2.28	1.93
query23	16.83	0.98	0.88
query24	3.20	1.55	1.19
query25	0.09	0.07	0.08
query26	0.63	0.15	0.14
query27	0.05	0.04	0.04
query28	10.04	1.12	1.07
query29	12.57	3.26	3.26
query30	0.25	0.05	0.05
query31	2.85	0.37	0.39
query32	3.28	0.46	0.48
query33	2.98	3.04	3.01
query34	16.85	4.48	4.48
query35	4.45	4.44	4.46
query36	0.66	0.48	0.49
query37	0.09	0.05	0.06
query38	0.05	0.04	0.04
query39	0.04	0.02	0.02
query40	0.15	0.12	0.12
query41	0.08	0.02	0.03
query42	0.04	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 106.65 s
Total hot run time: 33.38 s

@bobhan1
Copy link
Contributor Author

bobhan1 commented Jan 3, 2025

run p0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants