WIP: Validator #8151

bm1549 · 2025-01-03T19:21:19Z

What Does This Do

Creates a validator to make sure classes behave as expected

Motivation

Additional Notes

Current output

./dd-java-agent/instrumentation/vertx-mysql-client-3.9/src/main/java/datadog/trace/instrumentation/vertx_sql_client/SqlClientBaseAdvice.java: Class 'SqlClientBaseAdvice' does not have a method tagged with an @Advice annotation.
./dd-java-agent/instrumentation/vertx-mysql-client-3.9/src/main/java/datadog/trace/instrumentation/vertx_sql_client/QueryAdvice.java: Class 'QueryAdvice' does not have a method tagged with an @Advice annotation.
./dd-java-agent/instrumentation/aws-lambda-handler/src/main/java/datadog/trace/instrumentation/aws/v1/lambda/LambdaHandlerDecorator.java: Class 'LambdaHandlerDecorator' should extend a class with 'Decorator' in its name.
./dd-java-agent/instrumentation/vertx-mysql-client-4.0/src/main/java/datadog/trace/instrumentation/vertx_sql_client_4/SqlClientBaseAdvice.java: Class 'SqlClientBaseAdvice' does not have a method tagged with an @Advice annotation.
./dd-java-agent/instrumentation/vertx-mysql-client-4.0/src/main/java/datadog/trace/instrumentation/vertx_sql_client_4/QueryAdvice.java: Class 'QueryAdvice' does not have a method tagged with an @Advice annotation.
./dd-java-agent/instrumentation/netty-3.8/src/main/java/datadog/trace/instrumentation/netty38/AbstractNettyAdvice.java: Class 'AbstractNettyAdvice' does not have a method tagged with an @Advice annotation.
./dd-java-agent/instrumentation/aws-java-sdk-1.11.0/src/main/java/datadog/trace/instrumentation/aws/v0/AwsSdkModule.java: Class 'AwsSdkModule' extends a 'Instrument' class but does not end with 'Instrumentation'.
./dd-java-agent/instrumentation/spring-security-5/src/main/java/datadog/trace/instrumentation/springsecurity5/SpringSecurityUserEventDecorator.java: Class 'SpringSecurityUserEventDecorator' should extend a class with 'Decorator' in its name.

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

mcculls · 2025-01-03T19:33:28Z

./dd-java-agent/instrumentation/aws-java-sdk-1.11.0/src/main/java/datadog/trace/instrumentation/aws/v0/AwsSdkModule.java: Class 'AwsSdkModule' extends a 'Instrument' class but does not end with 'Instrumentation'.

Note AwsSdkModule is not meant to end with Instrumentation because it's an InstrumentationModule which doesn't add any advice itself but collects other instrumentations - instead it should end in Module.

One way to capture this is to say that any class that overrides typeInstrumentations() should end in Module - if it doesn't override typeInstrumentations() then fall back to the Instrumenter check.

bm1549 · 2025-01-03T19:35:32Z

./dd-java-agent/instrumentation/aws-java-sdk-1.11.0/src/main/java/datadog/trace/instrumentation/aws/v0/AwsSdkModule.java: Class 'AwsSdkModule' extends a 'Instrument' class but does not end with 'Instrumentation'.

Note AwsSdkModule is not meant to end with Instrumentation because it's an InstrumentationModule which doesn't add any advice itself but collects other instrumentations - instead it should end in Module.

One way to capture this is to say that any class that overrides typeInstrumentations() should end in Module - if it doesn't override typeInstrumentations() then fall back to the Instrumenter check.

@mcculls to simplify a bit, wdyt about just adding Module to the acceptable endings for classes that extend Instrument?

mcculls · 2025-01-03T19:49:45Z

@mcculls to simplify a bit, wdyt about just adding Module to the acceptable endings for classes that extend Instrument?

Sure, that's also OK

pr-commenter · 2025-01-03T19:59:42Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	bm1549/validator
git_commit_date	1735897629	1735934056
git_commit_sha	`a8b33d5`	`31d07da`
release_version	1.45.0-SNAPSHOT~a8b33d5c49	1.45.0-SNAPSHOT~31d07dacca

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1735936460	1735936460
ci_job_id	754372849	754372849
ci_pipeline_id	52141385	52141385
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 59 metrics, 4 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.45.0-SNAPSHOT~31d07dacca, baseline=1.45.0-SNAPSHOT~a8b33d5c49

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.057 s) : 0, 1056534
Total [baseline] (10.43 s) : 0, 10429621
Agent [candidate] (1.052 s) : 0, 1051680
Total [candidate] (10.463 s) : 0, 10462697
section appsec
Agent [baseline] (1.192 s) : 0, 1192117
Total [baseline] (10.702 s) : 0, 10702227
Agent [candidate] (1.19 s) : 0, 1189836
Total [candidate] (10.696 s) : 0, 10696277
section iast
Agent [baseline] (1.184 s) : 0, 1184171
Total [baseline] (10.968 s) : 0, 10967802
Agent [candidate] (1.192 s) : 0, 1191934
Total [candidate] (11.003 s) : 0, 11003435
section profiling
Agent [baseline] (1.271 s) : 0, 1270865
Total [baseline] (10.837 s) : 0, 10837328
Agent [candidate] (1.27 s) : 0, 1269791
Total [candidate] (10.762 s) : 0, 10761691

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.057 s	-
Agent	appsec	1.192 s	135.583 ms (12.8%)
Agent	iast	1.184 s	127.637 ms (12.1%)
Agent	profiling	1.271 s	214.331 ms (20.3%)
Total	tracing	10.43 s	-
Total	appsec	10.702 s	272.605 ms (2.6%)
Total	iast	10.968 s	538.181 ms (5.2%)
Total	profiling	10.837 s	407.707 ms (3.9%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.052 s	-
Agent	appsec	1.19 s	138.156 ms (13.1%)
Agent	iast	1.192 s	140.255 ms (13.3%)
Agent	profiling	1.27 s	218.111 ms (20.7%)
Total	tracing	10.463 s	-
Total	appsec	10.696 s	233.58 ms (2.2%)
Total	iast	11.003 s	540.738 ms (5.2%)
Total	profiling	10.762 s	298.994 ms (2.9%)

gantt
    title petclinic - break down per module: candidate=1.45.0-SNAPSHOT~31d07dacca, baseline=1.45.0-SNAPSHOT~a8b33d5c49

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (715.099 ms) : 0, 715099
BytebuddyAgent [candidate] (712.372 ms) : 0, 712372
GlobalTracer [baseline] (257.148 ms) : 0, 257148
GlobalTracer [candidate] (256.245 ms) : 0, 256245
AppSec [baseline] (55.176 ms) : 0, 55176
AppSec [candidate] (55.252 ms) : 0, 55252
Remote Config [baseline] (714.841 µs) : 0, 715
Remote Config [candidate] (724.967 µs) : 0, 725
Telemetry [baseline] (13.26 ms) : 0, 13260
Telemetry [candidate] (12.124 ms) : 0, 12124
section appsec
BytebuddyAgent [baseline] (733.185 ms) : 0, 733185
BytebuddyAgent [candidate] (731.221 ms) : 0, 731221
GlobalTracer [baseline] (254.202 ms) : 0, 254202
GlobalTracer [candidate] (254.094 ms) : 0, 254094
AppSec [baseline] (171.362 ms) : 0, 171362
AppSec [candidate] (170.926 ms) : 0, 170926
Remote Config [baseline] (660.321 µs) : 0, 660
Remote Config [candidate] (671.942 µs) : 0, 672
Telemetry [baseline] (8.027 ms) : 0, 8027
Telemetry [candidate] (8.055 ms) : 0, 8055
IAST [baseline] (19.27 ms) : 0, 19270
IAST [candidate] (19.568 ms) : 0, 19568
section iast
BytebuddyAgent [baseline] (833.183 ms) : 0, 833183
BytebuddyAgent [candidate] (838.356 ms) : 0, 838356
GlobalTracer [baseline] (247.06 ms) : 0, 247060
GlobalTracer [candidate] (249.336 ms) : 0, 249336
AppSec [baseline] (57.987 ms) : 0, 57987
AppSec [candidate] (58.378 ms) : 0, 58378
Remote Config [baseline] (667.023 µs) : 0, 667
Remote Config [candidate] (656.859 µs) : 0, 657
Telemetry [baseline] (8.617 ms) : 0, 8617
Telemetry [candidate] (8.614 ms) : 0, 8614
IAST [baseline] (21.551 ms) : 0, 21551
IAST [candidate] (21.39 ms) : 0, 21390
section profiling
ProfilingAgent [baseline] (95.531 ms) : 0, 95531
ProfilingAgent [candidate] (94.781 ms) : 0, 94781
BytebuddyAgent [baseline] (700.384 ms) : 0, 700384
BytebuddyAgent [candidate] (700.937 ms) : 0, 700937
GlobalTracer [baseline] (370.802 ms) : 0, 370802
GlobalTracer [candidate] (370.525 ms) : 0, 370525
AppSec [baseline] (53.979 ms) : 0, 53979
AppSec [candidate] (53.482 ms) : 0, 53482
Remote Config [baseline] (651.481 µs) : 0, 651
Remote Config [candidate] (651.762 µs) : 0, 652
Telemetry [baseline] (7.813 ms) : 0, 7813
Telemetry [candidate] (7.695 ms) : 0, 7695
Profiling [baseline] (95.555 ms) : 0, 95555
Profiling [candidate] (94.805 ms) : 0, 94805

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.45.0-SNAPSHOT~31d07dacca, baseline=1.45.0-SNAPSHOT~a8b33d5c49

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.057 s) : 0, 1057250
Total [baseline] (8.635 s) : 0, 8635241
Agent [candidate] (1.06 s) : 0, 1059557
Total [candidate] (8.622 s) : 0, 8622256
section iast
Agent [baseline] (1.18 s) : 0, 1179878
Total [baseline] (9.192 s) : 0, 9191551
Agent [candidate] (1.191 s) : 0, 1191058
Total [candidate] (9.24 s) : 0, 9240475
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.179 s) : 0, 1179327
Total [baseline] (9.186 s) : 0, 9185643
Agent [candidate] (1.177 s) : 0, 1177202
Total [candidate] (9.139 s) : 0, 9138811
section iast_TELEMETRY_OFF
Agent [baseline] (1.176 s) : 0, 1176427
Total [baseline] (9.174 s) : 0, 9173521
Agent [candidate] (1.177 s) : 0, 1177303
Total [candidate] (9.203 s) : 0, 9202524

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.057 s	-
Agent	iast	1.18 s	122.629 ms (11.6%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.179 s	122.078 ms (11.5%)
Agent	iast_TELEMETRY_OFF	1.176 s	119.178 ms (11.3%)
Total	tracing	8.635 s	-
Total	iast	9.192 s	556.31 ms (6.4%)
Total	iast_HARDCODED_SECRET_DISABLED	9.186 s	550.402 ms (6.4%)
Total	iast_TELEMETRY_OFF	9.174 s	538.28 ms (6.2%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.06 s	-
Agent	iast	1.191 s	131.501 ms (12.4%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.177 s	117.645 ms (11.1%)
Agent	iast_TELEMETRY_OFF	1.177 s	117.746 ms (11.1%)
Total	tracing	8.622 s	-
Total	iast	9.24 s	618.219 ms (7.2%)
Total	iast_HARDCODED_SECRET_DISABLED	9.139 s	516.555 ms (6.0%)
Total	iast_TELEMETRY_OFF	9.203 s	580.268 ms (6.7%)

gantt
    title insecure-bank - break down per module: candidate=1.45.0-SNAPSHOT~31d07dacca, baseline=1.45.0-SNAPSHOT~a8b33d5c49

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (716.054 ms) : 0, 716054
BytebuddyAgent [candidate] (717.189 ms) : 0, 717189
GlobalTracer [baseline] (257.356 ms) : 0, 257356
GlobalTracer [candidate] (257.725 ms) : 0, 257725
AppSec [baseline] (55.553 ms) : 0, 55553
AppSec [candidate] (55.29 ms) : 0, 55290
Remote Config [baseline] (717.301 µs) : 0, 717
Remote Config [candidate] (734.467 µs) : 0, 734
Telemetry [baseline] (12.618 ms) : 0, 12618
Telemetry [candidate] (13.519 ms) : 0, 13519
section iast
BytebuddyAgent [baseline] (829.733 ms) : 0, 829733
BytebuddyAgent [candidate] (837.671 ms) : 0, 837671
GlobalTracer [baseline] (246.694 ms) : 0, 246694
GlobalTracer [candidate] (248.683 ms) : 0, 248683
AppSec [baseline] (57.923 ms) : 0, 57923
AppSec [candidate] (58.48 ms) : 0, 58480
IAST [baseline] (21.274 ms) : 0, 21274
IAST [candidate] (21.723 ms) : 0, 21723
Remote Config [baseline] (641.923 µs) : 0, 642
Remote Config [candidate] (660.022 µs) : 0, 660
Telemetry [baseline] (8.554 ms) : 0, 8554
Telemetry [candidate] (8.702 ms) : 0, 8702
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (828.709 ms) : 0, 828709
BytebuddyAgent [candidate] (827.619 ms) : 0, 827619
GlobalTracer [baseline] (246.767 ms) : 0, 246767
GlobalTracer [candidate] (246.321 ms) : 0, 246321
AppSec [baseline] (58.137 ms) : 0, 58137
AppSec [candidate] (57.916 ms) : 0, 57916
IAST [baseline] (21.413 ms) : 0, 21413
IAST [candidate] (21.134 ms) : 0, 21134
Remote Config [baseline] (659.45 µs) : 0, 659
Remote Config [candidate] (677.498 µs) : 0, 677
Telemetry [baseline] (8.672 ms) : 0, 8672
Telemetry [candidate] (8.522 ms) : 0, 8522
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (826.585 ms) : 0, 826585
BytebuddyAgent [candidate] (826.791 ms) : 0, 826791
GlobalTracer [baseline] (246.841 ms) : 0, 246841
GlobalTracer [candidate] (246.851 ms) : 0, 246851
AppSec [baseline] (57.821 ms) : 0, 57821
AppSec [candidate] (58.239 ms) : 0, 58239
IAST [baseline] (21.088 ms) : 0, 21088
IAST [candidate] (21.25 ms) : 0, 21250
Remote Config [baseline] (649.568 µs) : 0, 650
Remote Config [candidate] (662.408 µs) : 0, 662
Telemetry [baseline] (8.446 ms) : 0, 8446
Telemetry [candidate] (8.509 ms) : 0, 8509

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2025-01-03T20:02:28	2025-01-03T20:09:23
git_branch	master	bm1549/validator
git_commit_date	1735897629	1735934056
git_commit_sha	`a8b33d5`	`31d07da`
release_version	1.45.0-SNAPSHOT~a8b33d5c49	1.45.0-SNAPSHOT~31d07dacca
start_time	2025-01-03T20:02:15	2025-01-03T20:09:10

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1735935314	1735935314
ci_job_id	754372850	754372850
ci_pipeline_id	52141385	52141385
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	iast	iast

Summary

Found 0 performance improvements and 1 performance regressions! Performance is the same for 10 metrics, 17 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:petclinic:profiling	worse [+31.565µs; +85.009µs] or [+2.123%; +5.718%]	unstable [-554.485op/s; +554.485op/s] or [-18.021%; +18.021%]	1.545ms	3076.923op/s	1.487ms	3076.923op/s

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.45.0-SNAPSHOT~31d07dacca, baseline=1.45.0-SNAPSHOT~a8b33d5c49
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.339 ms) : 1319, 1359
.   : milestone, 1339,
appsec (1.725 ms) : 1702, 1748
.   : milestone, 1725,
appsec_no_iast (1.73 ms) : 1705, 1755
.   : milestone, 1730,
iast (1.505 ms) : 1482, 1528
.   : milestone, 1505,
profiling (1.487 ms) : 1463, 1510
.   : milestone, 1487,
tracing (1.466 ms) : 1441, 1490
.   : milestone, 1466,
section candidate
no_agent (1.342 ms) : 1323, 1362
.   : milestone, 1342,
appsec (1.73 ms) : 1706, 1753
.   : milestone, 1730,
appsec_no_iast (1.759 ms) : 1737, 1781
.   : milestone, 1759,
iast (1.488 ms) : 1465, 1510
.   : milestone, 1488,
profiling (1.545 ms) : 1519, 1571
.   : milestone, 1545,
tracing (1.454 ms) : 1429, 1479
.   : milestone, 1454,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.339 ms [1.319 ms, 1.359 ms]	-
appsec	1.725 ms [1.702 ms, 1.748 ms]	386.135 µs (28.8%)
appsec_no_iast	1.73 ms [1.705 ms, 1.755 ms]	391.401 µs (29.2%)
iast	1.505 ms [1.482 ms, 1.528 ms]	166.383 µs (12.4%)
profiling	1.487 ms [1.463 ms, 1.51 ms]	147.902 µs (11.0%)
tracing	1.466 ms [1.441 ms, 1.49 ms]	126.82 µs (9.5%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.342 ms [1.323 ms, 1.362 ms]	-
appsec	1.73 ms [1.706 ms, 1.753 ms]	387.41 µs (28.9%)
appsec_no_iast	1.759 ms [1.737 ms, 1.781 ms]	416.713 µs (31.0%)
iast	1.488 ms [1.465 ms, 1.51 ms]	145.345 µs (10.8%)
profiling	1.545 ms [1.519 ms, 1.571 ms]	202.784 µs (15.1%)
tracing	1.454 ms [1.429 ms, 1.479 ms]	111.693 µs (8.3%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.45.0-SNAPSHOT~31d07dacca, baseline=1.45.0-SNAPSHOT~a8b33d5c49
    dateFormat X
    axisFormat %s
section baseline
no_agent (369.995 µs) : 350, 390
.   : milestone, 370,
iast (487.784 µs) : 466, 510
.   : milestone, 488,
iast_FULL (650.952 µs) : 629, 673
.   : milestone, 651,
iast_GLOBAL (519.256 µs) : 497, 541
.   : milestone, 519,
iast_HARDCODED_SECRET_DISABLED (487.141 µs) : 466, 509
.   : milestone, 487,
iast_INACTIVE (444.578 µs) : 424, 466
.   : milestone, 445,
iast_TELEMETRY_OFF (477.253 µs) : 455, 499
.   : milestone, 477,
tracing (446.9 µs) : 426, 468
.   : milestone, 447,
section candidate
no_agent (372.456 µs) : 353, 392
.   : milestone, 372,
iast (490.391 µs) : 469, 512
.   : milestone, 490,
iast_FULL (650.731 µs) : 629, 672
.   : milestone, 651,
iast_GLOBAL (513.898 µs) : 492, 536
.   : milestone, 514,
iast_HARDCODED_SECRET_DISABLED (484.432 µs) : 463, 506
.   : milestone, 484,
iast_INACTIVE (441.893 µs) : 421, 463
.   : milestone, 442,
iast_TELEMETRY_OFF (475.002 µs) : 454, 496
.   : milestone, 475,
tracing (443.354 µs) : 422, 464
.   : milestone, 443,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	369.995 µs [350.371 µs, 389.618 µs]	-
iast	487.784 µs [465.802 µs, 509.765 µs]	117.789 µs (31.8%)
iast_FULL	650.952 µs [629.332 µs, 672.573 µs]	280.957 µs (75.9%)
iast_GLOBAL	519.256 µs [497.301 µs, 541.21 µs]	149.261 µs (40.3%)
iast_HARDCODED_SECRET_DISABLED	487.141 µs [465.665 µs, 508.617 µs]	117.146 µs (31.7%)
iast_INACTIVE	444.578 µs [423.55 µs, 465.606 µs]	74.583 µs (20.2%)
iast_TELEMETRY_OFF	477.253 µs [455.08 µs, 499.425 µs]	107.258 µs (29.0%)
tracing	446.9 µs [425.575 µs, 468.226 µs]	76.905 µs (20.8%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	372.456 µs [352.742 µs, 392.17 µs]	-
iast	490.391 µs [468.766 µs, 512.016 µs]	117.935 µs (31.7%)
iast_FULL	650.731 µs [629.075 µs, 672.388 µs]	278.275 µs (74.7%)
iast_GLOBAL	513.898 µs [491.851 µs, 535.945 µs]	141.442 µs (38.0%)
iast_HARDCODED_SECRET_DISABLED	484.432 µs [462.908 µs, 505.956 µs]	111.975 µs (30.1%)
iast_INACTIVE	441.893 µs [421.193 µs, 462.593 µs]	69.437 µs (18.6%)
iast_TELEMETRY_OFF	475.002 µs [453.511 µs, 496.494 µs]	102.546 µs (27.5%)
tracing	443.354 µs [422.369 µs, 464.34 µs]	70.898 µs (19.0%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	bm1549/validator
git_commit_date	1735897629	1735934056
git_commit_sha	`a8b33d5`	`31d07da`
release_version	1.45.0-SNAPSHOT~a8b33d5c49	1.45.0-SNAPSHOT~31d07dacca

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1735936005	1735936005
ci_job_id	754372851	754372851
ci_pipeline_id	52141385	52141385
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.45.0-SNAPSHOT~31d07dacca, baseline=1.45.0-SNAPSHOT~a8b33d5c49
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.468 ms) : 1457, 1480
.   : milestone, 1468,
appsec (2.352 ms) : 2310, 2394
.   : milestone, 2352,
iast (2.103 ms) : 2050, 2157
.   : milestone, 2103,
iast_GLOBAL (2.139 ms) : 2085, 2192
.   : milestone, 2139,
profiling (1.997 ms) : 1951, 2042
.   : milestone, 1997,
tracing (1.932 ms) : 1892, 1973
.   : milestone, 1932,
section candidate
no_agent (1.465 ms) : 1454, 1476
.   : milestone, 1465,
appsec (2.346 ms) : 2304, 2388
.   : milestone, 2346,
iast (2.091 ms) : 2038, 2144
.   : milestone, 2091,
iast_GLOBAL (2.128 ms) : 2075, 2181
.   : milestone, 2128,
profiling (1.963 ms) : 1919, 2006
.   : milestone, 1963,
tracing (1.924 ms) : 1884, 1965
.   : milestone, 1924,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.468 ms [1.457 ms, 1.48 ms]	-
appsec	2.352 ms [2.31 ms, 2.394 ms]	883.54 µs (60.2%)
iast	2.103 ms [2.05 ms, 2.157 ms]	634.975 µs (43.2%)
iast_GLOBAL	2.139 ms [2.085 ms, 2.192 ms]	670.35 µs (45.7%)
profiling	1.997 ms [1.951 ms, 2.042 ms]	528.319 µs (36.0%)
tracing	1.932 ms [1.892 ms, 1.973 ms]	464.143 µs (31.6%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.465 ms [1.454 ms, 1.476 ms]	-
appsec	2.346 ms [2.304 ms, 2.388 ms]	880.935 µs (60.1%)
iast	2.091 ms [2.038 ms, 2.144 ms]	626.124 µs (42.7%)
iast_GLOBAL	2.128 ms [2.075 ms, 2.181 ms]	662.833 µs (45.2%)
profiling	1.963 ms [1.919 ms, 2.006 ms]	497.792 µs (34.0%)
tracing	1.924 ms [1.884 ms, 1.965 ms]	459.445 µs (31.4%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.45.0-SNAPSHOT~31d07dacca, baseline=1.45.0-SNAPSHOT~a8b33d5c49
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.736 s) : 14736000, 14736000
.   : milestone, 14736000,
appsec (14.966 s) : 14966000, 14966000
.   : milestone, 14966000,
iast (18.966 s) : 18966000, 18966000
.   : milestone, 18966000,
iast_GLOBAL (17.9 s) : 17900000, 17900000
.   : milestone, 17900000,
profiling (15.854 s) : 15854000, 15854000
.   : milestone, 15854000,
tracing (15.256 s) : 15256000, 15256000
.   : milestone, 15256000,
section candidate
no_agent (15.553 s) : 15553000, 15553000
.   : milestone, 15553000,
appsec (14.968 s) : 14968000, 14968000
.   : milestone, 14968000,
iast (18.888 s) : 18888000, 18888000
.   : milestone, 18888000,
iast_GLOBAL (18.031 s) : 18031000, 18031000
.   : milestone, 18031000,
profiling (14.849 s) : 14849000, 14849000
.   : milestone, 14849000,
tracing (15.153 s) : 15153000, 15153000
.   : milestone, 15153000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.736 s [14.736 s, 14.736 s]	-
appsec	14.966 s [14.966 s, 14.966 s]	230.0 ms (1.6%)
iast	18.966 s [18.966 s, 18.966 s]	4.23 s (28.7%)
iast_GLOBAL	17.9 s [17.9 s, 17.9 s]	3.164 s (21.5%)
profiling	15.854 s [15.854 s, 15.854 s]	1.118 s (7.6%)
tracing	15.256 s [15.256 s, 15.256 s]	520.0 ms (3.5%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.553 s [15.553 s, 15.553 s]	-
appsec	14.968 s [14.968 s, 14.968 s]	-585.0 ms (-3.8%)
iast	18.888 s [18.888 s, 18.888 s]	3.335 s (21.4%)
iast_GLOBAL	18.031 s [18.031 s, 18.031 s]	2.478 s (15.9%)
profiling	14.849 s [14.849 s, 14.849 s]	-704.0 ms (-4.5%)
tracing	15.153 s [15.153 s, 15.153 s]	-400.0 ms (-2.6%)

PerfectSlayer

Side question: Why does it revert static import where I usually recommend using them?

bm1549 · 2025-01-06T20:52:58Z

Side question: Why does it revert static import where I usually recommend using them?

@PerfectSlayer the validator doesn't fix anything, it just logs the issues it sees. All of the fixes were done by myself

As for the import change, that's because the script looks for Advice. annotations, which weren't being picked up properly when the full name was imported

PerfectSlayer · 2025-01-07T09:15:38Z

which weren't being picked up properly when the full name was imported

Is it a limitation of the script/library you use? As I would really push in the other side during review (using static import) 🤔

WIP: Validator

23c4f00

modules are okay too

31d07da

PerfectSlayer reviewed Jan 6, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP: Validator #8151

WIP: Validator #8151

bm1549 commented Jan 3, 2025

mcculls commented Jan 3, 2025 •

edited

Loading

bm1549 commented Jan 3, 2025

mcculls commented Jan 3, 2025

pr-commenter bot commented Jan 3, 2025 •

edited

Loading

PerfectSlayer left a comment

bm1549 commented Jan 6, 2025

PerfectSlayer commented Jan 7, 2025

WIP: Validator #8151

Are you sure you want to change the base?

WIP: Validator #8151

Conversation

bm1549 commented Jan 3, 2025

What Does This Do

Motivation

Additional Notes

Contributor Checklist

mcculls commented Jan 3, 2025 • edited Loading

bm1549 commented Jan 3, 2025

mcculls commented Jan 3, 2025

pr-commenter bot commented Jan 3, 2025 • edited Loading

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

PerfectSlayer left a comment

Choose a reason for hiding this comment

bm1549 commented Jan 6, 2025

PerfectSlayer commented Jan 7, 2025

mcculls commented Jan 3, 2025 •

edited

Loading

pr-commenter bot commented Jan 3, 2025 •

edited

Loading