Exclude flaky errors from error log checks #8123

smola · 2024-12-20T16:04:08Z

What Does This Do

Follow up to #8111.

Various errors can appear more frequently than I anticipated. They need to be excluded.

Motivation

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

pr-commenter · 2024-12-20T16:45:22Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	smola/add-error-log-exclusions
git_commit_date	1734694675	1734712342
git_commit_sha	`a3e9bda`	`dac2889`
release_version	1.45.0-SNAPSHOT~a3e9bda406	1.45.0-SNAPSHOT~dac2889c6e

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1734714676	1734714676
ci_job_id	747246316	747246316
ci_pipeline_id	51665956	51665956
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 58 metrics, 5 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.45.0-SNAPSHOT~dac2889c6e, baseline=1.45.0-SNAPSHOT~a3e9bda406

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.053 s) : 0, 1053268
Total [baseline] (10.469 s) : 0, 10469468
Agent [candidate] (1.054 s) : 0, 1054202
Total [candidate] (10.388 s) : 0, 10388482
section appsec
Agent [baseline] (1.186 s) : 0, 1185640
Total [baseline] (10.749 s) : 0, 10749425
Agent [candidate] (1.184 s) : 0, 1183923
Total [candidate] (10.666 s) : 0, 10665736
section iast
Agent [baseline] (1.179 s) : 0, 1179149
Total [baseline] (10.907 s) : 0, 10906793
Agent [candidate] (1.177 s) : 0, 1177221
Total [candidate] (10.893 s) : 0, 10892683
section profiling
Agent [baseline] (1.277 s) : 0, 1277209
Total [baseline] (10.832 s) : 0, 10832094
Agent [candidate] (1.274 s) : 0, 1274477
Total [candidate] (10.83 s) : 0, 10830364

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.053 s	-
Agent	appsec	1.186 s	132.372 ms (12.6%)
Agent	iast	1.179 s	125.881 ms (12.0%)
Agent	profiling	1.277 s	223.941 ms (21.3%)
Total	tracing	10.469 s	-
Total	appsec	10.749 s	279.958 ms (2.7%)
Total	iast	10.907 s	437.325 ms (4.2%)
Total	profiling	10.832 s	362.626 ms (3.5%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.054 s	-
Agent	appsec	1.184 s	129.721 ms (12.3%)
Agent	iast	1.177 s	123.019 ms (11.7%)
Agent	profiling	1.274 s	220.275 ms (20.9%)
Total	tracing	10.388 s	-
Total	appsec	10.666 s	277.254 ms (2.7%)
Total	iast	10.893 s	504.201 ms (4.9%)
Total	profiling	10.83 s	441.882 ms (4.3%)

gantt
    title petclinic - break down per module: candidate=1.45.0-SNAPSHOT~dac2889c6e, baseline=1.45.0-SNAPSHOT~a3e9bda406

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (711.734 ms) : 0, 711734
BytebuddyAgent [candidate] (712.134 ms) : 0, 712134
GlobalTracer [baseline] (255.582 ms) : 0, 255582
GlobalTracer [candidate] (255.893 ms) : 0, 255893
AppSec [baseline] (58.46 ms) : 0, 58460
AppSec [candidate] (57.919 ms) : 0, 57919
Remote Config [baseline] (688.84 µs) : 0, 689
Remote Config [candidate] (695.356 µs) : 0, 695
Telemetry [baseline] (11.843 ms) : 0, 11843
Telemetry [candidate] (12.59 ms) : 0, 12590
section appsec
BytebuddyAgent [baseline] (728.754 ms) : 0, 728754
BytebuddyAgent [candidate] (727.465 ms) : 0, 727465
GlobalTracer [baseline] (252.532 ms) : 0, 252532
GlobalTracer [candidate] (252.221 ms) : 0, 252221
AppSec [baseline] (171.135 ms) : 0, 171135
AppSec [candidate] (171.091 ms) : 0, 171091
IAST [baseline] (19.311 ms) : 0, 19311
IAST [candidate] (19.292 ms) : 0, 19292
Remote Config [baseline] (662.21 µs) : 0, 662
Remote Config [candidate] (652.95 µs) : 0, 653
Telemetry [baseline] (7.992 ms) : 0, 7992
Telemetry [candidate] (7.941 ms) : 0, 7941
section iast
BytebuddyAgent [baseline] (829.747 ms) : 0, 829747
BytebuddyAgent [candidate] (828.085 ms) : 0, 828085
GlobalTracer [baseline] (246.603 ms) : 0, 246603
GlobalTracer [candidate] (245.895 ms) : 0, 245895
AppSec [baseline] (57.771 ms) : 0, 57771
AppSec [candidate] (57.738 ms) : 0, 57738
IAST [baseline] (21.118 ms) : 0, 21118
IAST [candidate] (21.412 ms) : 0, 21412
Remote Config [baseline] (649.223 µs) : 0, 649
Remote Config [candidate] (653.761 µs) : 0, 654
Telemetry [baseline] (8.441 ms) : 0, 8441
Telemetry [candidate] (8.408 ms) : 0, 8408
section profiling
BytebuddyAgent [baseline] (704.068 ms) : 0, 704068
BytebuddyAgent [candidate] (702.704 ms) : 0, 702704
GlobalTracer [baseline] (372.429 ms) : 0, 372429
GlobalTracer [candidate] (371.373 ms) : 0, 371373
AppSec [baseline] (54.057 ms) : 0, 54057
AppSec [candidate] (53.913 ms) : 0, 53913
Remote Config [baseline] (657.415 µs) : 0, 657
Remote Config [candidate] (665.877 µs) : 0, 666
Telemetry [baseline] (8.04 ms) : 0, 8040
Telemetry [candidate] (7.817 ms) : 0, 7817
ProfilingAgent [baseline] (96.151 ms) : 0, 96151
ProfilingAgent [candidate] (96.231 ms) : 0, 96231
Profiling [baseline] (96.177 ms) : 0, 96177
Profiling [candidate] (96.255 ms) : 0, 96255

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.45.0-SNAPSHOT~dac2889c6e, baseline=1.45.0-SNAPSHOT~a3e9bda406

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.052 s) : 0, 1052126
Total [baseline] (8.601 s) : 0, 8601049
Agent [candidate] (1.053 s) : 0, 1052799
Total [candidate] (8.603 s) : 0, 8602884
section iast
Agent [baseline] (1.178 s) : 0, 1178371
Total [baseline] (9.215 s) : 0, 9214894
Agent [candidate] (1.177 s) : 0, 1177417
Total [candidate] (9.231 s) : 0, 9231061
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.184 s) : 0, 1183964
Total [baseline] (9.171 s) : 0, 9171369
Agent [candidate] (1.177 s) : 0, 1176990
Total [candidate] (9.199 s) : 0, 9198978
section iast_TELEMETRY_OFF
Agent [baseline] (1.175 s) : 0, 1175041
Total [baseline] (9.171 s) : 0, 9170536
Agent [candidate] (1.172 s) : 0, 1171651
Total [candidate] (9.156 s) : 0, 9156307

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.052 s	-
Agent	iast	1.178 s	126.244 ms (12.0%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.184 s	131.837 ms (12.5%)
Agent	iast_TELEMETRY_OFF	1.175 s	122.915 ms (11.7%)
Total	tracing	8.601 s	-
Total	iast	9.215 s	613.845 ms (7.1%)
Total	iast_HARDCODED_SECRET_DISABLED	9.171 s	570.32 ms (6.6%)
Total	iast_TELEMETRY_OFF	9.171 s	569.487 ms (6.6%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.053 s	-
Agent	iast	1.177 s	124.618 ms (11.8%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.177 s	124.191 ms (11.8%)
Agent	iast_TELEMETRY_OFF	1.172 s	118.851 ms (11.3%)
Total	tracing	8.603 s	-
Total	iast	9.231 s	628.177 ms (7.3%)
Total	iast_HARDCODED_SECRET_DISABLED	9.199 s	596.094 ms (6.9%)
Total	iast_TELEMETRY_OFF	9.156 s	553.423 ms (6.4%)

gantt
    title insecure-bank - break down per module: candidate=1.45.0-SNAPSHOT~dac2889c6e, baseline=1.45.0-SNAPSHOT~a3e9bda406

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (711.099 ms) : 0, 711099
BytebuddyAgent [candidate] (711.243 ms) : 0, 711243
GlobalTracer [baseline] (255.09 ms) : 0, 255090
GlobalTracer [candidate] (256.172 ms) : 0, 256172
AppSec [baseline] (58.572 ms) : 0, 58572
AppSec [candidate] (57.985 ms) : 0, 57985
Remote Config [baseline] (687.341 µs) : 0, 687
Remote Config [candidate] (685.839 µs) : 0, 686
Telemetry [baseline] (11.814 ms) : 0, 11814
Telemetry [candidate] (11.791 ms) : 0, 11791
section iast
BytebuddyAgent [baseline] (827.898 ms) : 0, 827898
BytebuddyAgent [candidate] (828.049 ms) : 0, 828049
GlobalTracer [baseline] (247.086 ms) : 0, 247086
GlobalTracer [candidate] (246.43 ms) : 0, 246430
AppSec [baseline] (57.899 ms) : 0, 57899
AppSec [candidate] (57.59 ms) : 0, 57590
IAST [baseline] (21.346 ms) : 0, 21346
IAST [candidate] (21.272 ms) : 0, 21272
Remote Config [baseline] (683.88 µs) : 0, 684
Remote Config [candidate] (689.507 µs) : 0, 690
Telemetry [baseline] (8.55 ms) : 0, 8550
Telemetry [candidate] (8.459 ms) : 0, 8459
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (832.935 ms) : 0, 832935
BytebuddyAgent [candidate] (826.22 ms) : 0, 826220
GlobalTracer [baseline] (247.782 ms) : 0, 247782
GlobalTracer [candidate] (247.507 ms) : 0, 247507
AppSec [baseline] (58.035 ms) : 0, 58035
AppSec [candidate] (57.883 ms) : 0, 57883
IAST [baseline] (21.087 ms) : 0, 21087
IAST [candidate] (21.237 ms) : 0, 21237
Remote Config [baseline] (650.305 µs) : 0, 650
Remote Config [candidate] (655.379 µs) : 0, 655
Telemetry [baseline] (8.466 ms) : 0, 8466
Telemetry [candidate] (8.548 ms) : 0, 8548
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (825.695 ms) : 0, 825695
BytebuddyAgent [candidate] (823.346 ms) : 0, 823346
GlobalTracer [baseline] (246.959 ms) : 0, 246959
GlobalTracer [candidate] (246.345 ms) : 0, 246345
AppSec [baseline] (57.782 ms) : 0, 57782
AppSec [candidate] (57.609 ms) : 0, 57609
IAST [baseline] (20.788 ms) : 0, 20788
IAST [candidate] (20.485 ms) : 0, 20485
Remote Config [baseline] (647.555 µs) : 0, 648
Remote Config [candidate] (631.215 µs) : 0, 631
Telemetry [baseline] (8.315 ms) : 0, 8315
Telemetry [candidate] (8.266 ms) : 0, 8266

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2024-12-20T16:41:41	2024-12-20T16:48:41
git_branch	master	smola/add-error-log-exclusions
git_commit_date	1734694675	1734712342
git_commit_sha	`a3e9bda`	`dac2889`
release_version	1.45.0-SNAPSHOT~a3e9bda406	1.45.0-SNAPSHOT~dac2889c6e
start_time	2024-12-20T16:41:27	2024-12-20T16:48:28

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1734713677	1734713677
ci_job_id	747246318	747246318
ci_pipeline_id	51665956	51665956
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 16 unstable metrics.

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.45.0-SNAPSHOT~dac2889c6e, baseline=1.45.0-SNAPSHOT~a3e9bda406
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.366 ms) : 1347, 1385
.   : milestone, 1366,
appsec (1.752 ms) : 1729, 1776
.   : milestone, 1752,
appsec_no_iast (1.786 ms) : 1762, 1810
.   : milestone, 1786,
iast (1.508 ms) : 1484, 1531
.   : milestone, 1508,
profiling (1.53 ms) : 1504, 1556
.   : milestone, 1530,
tracing (1.486 ms) : 1461, 1512
.   : milestone, 1486,
section candidate
no_agent (1.363 ms) : 1343, 1383
.   : milestone, 1363,
appsec (1.765 ms) : 1742, 1789
.   : milestone, 1765,
appsec_no_iast (1.777 ms) : 1753, 1802
.   : milestone, 1777,
iast (1.511 ms) : 1487, 1534
.   : milestone, 1511,
profiling (1.562 ms) : 1537, 1587
.   : milestone, 1562,
tracing (1.492 ms) : 1467, 1516
.   : milestone, 1492,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.366 ms [1.347 ms, 1.385 ms]	-
appsec	1.752 ms [1.729 ms, 1.776 ms]	386.545 µs (28.3%)
appsec_no_iast	1.786 ms [1.762 ms, 1.81 ms]	420.111 µs (30.8%)
iast	1.508 ms [1.484 ms, 1.531 ms]	141.608 µs (10.4%)
profiling	1.53 ms [1.504 ms, 1.556 ms]	163.802 µs (12.0%)
tracing	1.486 ms [1.461 ms, 1.512 ms]	120.437 µs (8.8%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.363 ms [1.343 ms, 1.383 ms]	-
appsec	1.765 ms [1.742 ms, 1.789 ms]	402.322 µs (29.5%)
appsec_no_iast	1.777 ms [1.753 ms, 1.802 ms]	414.368 µs (30.4%)
iast	1.511 ms [1.487 ms, 1.534 ms]	147.706 µs (10.8%)
profiling	1.562 ms [1.537 ms, 1.587 ms]	198.916 µs (14.6%)
tracing	1.492 ms [1.467 ms, 1.516 ms]	128.53 µs (9.4%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.45.0-SNAPSHOT~dac2889c6e, baseline=1.45.0-SNAPSHOT~a3e9bda406
    dateFormat X
    axisFormat %s
section baseline
no_agent (380.751 µs) : 361, 401
.   : milestone, 381,
iast (491.591 µs) : 470, 513
.   : milestone, 492,
iast_FULL (662.172 µs) : 640, 684
.   : milestone, 662,
iast_GLOBAL (524.209 µs) : 503, 546
.   : milestone, 524,
iast_HARDCODED_SECRET_DISABLED (502.84 µs) : 482, 524
.   : milestone, 503,
iast_INACTIVE (461.657 µs) : 441, 483
.   : milestone, 462,
iast_TELEMETRY_OFF (484.971 µs) : 464, 506
.   : milestone, 485,
tracing (463.511 µs) : 441, 486
.   : milestone, 464,
section candidate
no_agent (381.307 µs) : 362, 401
.   : milestone, 381,
iast (500.085 µs) : 478, 522
.   : milestone, 500,
iast_FULL (658.818 µs) : 637, 680
.   : milestone, 659,
iast_GLOBAL (525.484 µs) : 504, 547
.   : milestone, 525,
iast_HARDCODED_SECRET_DISABLED (494.65 µs) : 473, 516
.   : milestone, 495,
iast_INACTIVE (465.555 µs) : 444, 487
.   : milestone, 466,
iast_TELEMETRY_OFF (488.647 µs) : 467, 510
.   : milestone, 489,
tracing (456.436 µs) : 435, 478
.   : milestone, 456,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	380.751 µs [360.856 µs, 400.646 µs]	-
iast	491.591 µs [470.361 µs, 512.821 µs]	110.84 µs (29.1%)
iast_FULL	662.172 µs [640.417 µs, 683.927 µs]	281.421 µs (73.9%)
iast_GLOBAL	524.209 µs [502.833 µs, 545.584 µs]	143.458 µs (37.7%)
iast_HARDCODED_SECRET_DISABLED	502.84 µs [481.568 µs, 524.112 µs]	122.089 µs (32.1%)
iast_INACTIVE	461.657 µs [440.76 µs, 482.554 µs]	80.906 µs (21.2%)
iast_TELEMETRY_OFF	484.971 µs [463.527 µs, 506.415 µs]	104.22 µs (27.4%)
tracing	463.511 µs [440.784 µs, 486.238 µs]	82.76 µs (21.7%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	381.307 µs [361.687 µs, 400.928 µs]	-
iast	500.085 µs [478.241 µs, 521.93 µs]	118.778 µs (31.2%)
iast_FULL	658.818 µs [637.161 µs, 680.475 µs]	277.511 µs (72.8%)
iast_GLOBAL	525.484 µs [504.067 µs, 546.9 µs]	144.176 µs (37.8%)
iast_HARDCODED_SECRET_DISABLED	494.65 µs [473.422 µs, 515.879 µs]	113.343 µs (29.7%)
iast_INACTIVE	465.555 µs [443.73 µs, 487.379 µs]	84.247 µs (22.1%)
iast_TELEMETRY_OFF	488.647 µs [467.219 µs, 510.074 µs]	107.339 µs (28.2%)
tracing	456.436 µs [435.18 µs, 477.693 µs]	75.129 µs (19.7%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	smola/add-error-log-exclusions
git_commit_date	1734694675	1734712342
git_commit_sha	`a3e9bda`	`dac2889`
release_version	1.45.0-SNAPSHOT~a3e9bda406	1.45.0-SNAPSHOT~dac2889c6e

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1734714349	1734714349
ci_job_id	747246319	747246319
ci_pipeline_id	51665956	51665956
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.45.0-SNAPSHOT~dac2889c6e, baseline=1.45.0-SNAPSHOT~a3e9bda406
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.463 ms) : 1452, 1474
.   : milestone, 1463,
appsec (2.341 ms) : 2299, 2382
.   : milestone, 2341,
iast (2.088 ms) : 2034, 2141
.   : milestone, 2088,
iast_GLOBAL (2.136 ms) : 2082, 2190
.   : milestone, 2136,
profiling (1.997 ms) : 1950, 2043
.   : milestone, 1997,
tracing (1.934 ms) : 1893, 1975
.   : milestone, 1934,
section candidate
no_agent (1.466 ms) : 1455, 1477
.   : milestone, 1466,
appsec (2.344 ms) : 2302, 2386
.   : milestone, 2344,
iast (2.089 ms) : 2036, 2142
.   : milestone, 2089,
iast_GLOBAL (2.126 ms) : 2073, 2179
.   : milestone, 2126,
profiling (1.95 ms) : 1908, 1993
.   : milestone, 1950,
tracing (1.932 ms) : 1891, 1972
.   : milestone, 1932,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.463 ms [1.452 ms, 1.474 ms]	-
appsec	2.341 ms [2.299 ms, 2.382 ms]	877.462 µs (60.0%)
iast	2.088 ms [2.034 ms, 2.141 ms]	624.567 µs (42.7%)
iast_GLOBAL	2.136 ms [2.082 ms, 2.19 ms]	672.752 µs (46.0%)
profiling	1.997 ms [1.95 ms, 2.043 ms]	533.373 µs (36.5%)
tracing	1.934 ms [1.893 ms, 1.975 ms]	471.162 µs (32.2%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.466 ms [1.455 ms, 1.477 ms]	-
appsec	2.344 ms [2.302 ms, 2.386 ms]	877.917 µs (59.9%)
iast	2.089 ms [2.036 ms, 2.142 ms]	622.71 µs (42.5%)
iast_GLOBAL	2.126 ms [2.073 ms, 2.179 ms]	660.094 µs (45.0%)
profiling	1.95 ms [1.908 ms, 1.993 ms]	484.193 µs (33.0%)
tracing	1.932 ms [1.891 ms, 1.972 ms]	465.702 µs (31.8%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.45.0-SNAPSHOT~dac2889c6e, baseline=1.45.0-SNAPSHOT~a3e9bda406
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.694 s) : 14694000, 14694000
.   : milestone, 14694000,
appsec (14.953 s) : 14953000, 14953000
.   : milestone, 14953000,
iast (18.684 s) : 18684000, 18684000
.   : milestone, 18684000,
iast_GLOBAL (17.839 s) : 17839000, 17839000
.   : milestone, 17839000,
profiling (15.849 s) : 15849000, 15849000
.   : milestone, 15849000,
tracing (15.012 s) : 15012000, 15012000
.   : milestone, 15012000,
section candidate
no_agent (15.226 s) : 15226000, 15226000
.   : milestone, 15226000,
appsec (15.056 s) : 15056000, 15056000
.   : milestone, 15056000,
iast (18.908 s) : 18908000, 18908000
.   : milestone, 18908000,
iast_GLOBAL (18.019 s) : 18019000, 18019000
.   : milestone, 18019000,
profiling (14.922 s) : 14922000, 14922000
.   : milestone, 14922000,
tracing (15.152 s) : 15152000, 15152000
.   : milestone, 15152000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.694 s [14.694 s, 14.694 s]	-
appsec	14.953 s [14.953 s, 14.953 s]	259.0 ms (1.8%)
iast	18.684 s [18.684 s, 18.684 s]	3.99 s (27.2%)
iast_GLOBAL	17.839 s [17.839 s, 17.839 s]	3.145 s (21.4%)
profiling	15.849 s [15.849 s, 15.849 s]	1.155 s (7.9%)
tracing	15.012 s [15.012 s, 15.012 s]	318.0 ms (2.2%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.226 s [15.226 s, 15.226 s]	-
appsec	15.056 s [15.056 s, 15.056 s]	-170.0 ms (-1.1%)
iast	18.908 s [18.908 s, 18.908 s]	3.682 s (24.2%)
iast_GLOBAL	18.019 s [18.019 s, 18.019 s]	2.793 s (18.3%)
profiling	14.922 s [14.922 s, 14.922 s]	-304.0 ms (-2.0%)
tracing	15.152 s [15.152 s, 15.152 s]	-74.0 ms (-0.5%)

smola added comp: testing Testing tag: no release notes Changes to exclude from release notes run-tests: all Run all tests labels Dec 20, 2024

Exclude flaky errors from error log checks

dac2889

smola force-pushed the smola/add-error-log-exclusions branch from bc6f306 to dac2889 Compare December 20, 2024 16:32

smola mentioned this pull request Dec 20, 2024

Revert "Check logs for errors at smoke tests cleanup (#8111)" #8125

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Exclude flaky errors from error log checks #8123

Exclude flaky errors from error log checks #8123

smola commented Dec 20, 2024 •

edited

Loading

pr-commenter bot commented Dec 20, 2024 •

edited

Loading

Exclude flaky errors from error log checks #8123

Are you sure you want to change the base?

Exclude flaky errors from error log checks #8123

Conversation

smola commented Dec 20, 2024 • edited Loading

What Does This Do

Motivation

Additional Notes

Contributor Checklist

pr-commenter bot commented Dec 20, 2024 • edited Loading

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

smola commented Dec 20, 2024 •

edited

Loading

pr-commenter bot commented Dec 20, 2024 •

edited

Loading