Antithesis poc #9993

robertomonteromiguel · 2025-11-18T09:51:40Z

What Does This Do

Add Antithesis assertions in the tracer

Motivation

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any useful labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

pr-commenter · 2025-11-18T11:59:32Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	robertomonteromiguel/antithesis_poc_java_tracer
git_commit_date	1763454917	1763631063
git_commit_sha	`8d53878`	`e750f61`
release_version	1.56.0-SNAPSHOT~8d538789b9	1.56.0-SNAPSHOT~e750f61cfc

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1763633126	1763633126
ci_job_id	1246987901	1246987901
ci_pipeline_id	83313668	83313668
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-xqsgjrpu 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-xqsgjrpu 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 4 performance regressions! Performance is the same for 55 metrics, 6 unstable metrics.

scenario	Δ mean execution_time	candidate mean execution_time	baseline mean execution_time
scenario:startup:petclinic:profiling:ProfilingAgent	worse [+2.161ms; +4.100ms] or [+2.219%; +4.210%]	100.506ms	97.375ms
scenario:startup:petclinic:profiling:Telemetry	worse [+232.904µs; +575.283µs] or [+2.947%; +7.278%]	8.308ms	7.904ms
scenario:startup:petclinic:profiling:Flare Poller	worse [+89.139µs; +273.929µs] or [+2.420%; +7.435%]	3.866ms	3.684ms
scenario:startup:petclinic:profiling:Profiling	worse [+2.184ms; +4.138ms] or [+2.230%; +4.224%]	101.126ms	97.965ms

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.56.0-SNAPSHOT~e750f61cfc, baseline=1.56.0-SNAPSHOT~8d538789b9

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.114 s) : 0, 1113820
Total [baseline] (8.895 s) : 0, 8894964
Agent [candidate] (1.098 s) : 0, 1098100
Total [candidate] (8.752 s) : 0, 8751745
section iast
Agent [baseline] (1.234 s) : 0, 1234061
Total [baseline] (9.538 s) : 0, 9537955
Agent [candidate] (1.234 s) : 0, 1234401
Total [candidate] (9.436 s) : 0, 9436123

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.114 s	-
Agent	iast	1.234 s	120.241 ms (10.8%)
Total	tracing	8.895 s	-
Total	iast	9.538 s	642.991 ms (7.2%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.098 s	-
Agent	iast	1.234 s	136.301 ms (12.4%)
Total	tracing	8.752 s	-
Total	iast	9.436 s	684.378 ms (7.8%)

gantt
    title insecure-bank - break down per module: candidate=1.56.0-SNAPSHOT~e750f61cfc, baseline=1.56.0-SNAPSHOT~8d538789b9

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.478 ms) : 0, 1478
crashtracking [candidate] (1.449 ms) : 0, 1449
BytebuddyAgent [baseline] (714.826 ms) : 0, 714826
BytebuddyAgent [candidate] (704.425 ms) : 0, 704425
GlobalTracer [baseline] (252.105 ms) : 0, 252105
GlobalTracer [candidate] (248.982 ms) : 0, 248982
AppSec [baseline] (32.919 ms) : 0, 32919
AppSec [candidate] (32.352 ms) : 0, 32352
Debugger [baseline] (64.375 ms) : 0, 64375
Debugger [candidate] (63.25 ms) : 0, 63250
Remote Config [baseline] (682.932 µs) : 0, 683
Remote Config [candidate] (628.386 µs) : 0, 628
Telemetry [baseline] (8.39 ms) : 0, 8390
Telemetry [candidate] (8.398 ms) : 0, 8398
Flare Poller [baseline] (3.82 ms) : 0, 3820
Flare Poller [candidate] (3.653 ms) : 0, 3653
section iast
crashtracking [baseline] (1.445 ms) : 0, 1445
crashtracking [candidate] (1.447 ms) : 0, 1447
BytebuddyAgent [baseline] (827.304 ms) : 0, 827304
BytebuddyAgent [candidate] (826.767 ms) : 0, 826767
GlobalTracer [baseline] (237.589 ms) : 0, 237589
GlobalTracer [candidate] (238.212 ms) : 0, 238212
IAST [baseline] (29.297 ms) : 0, 29297
IAST [candidate] (26.593 ms) : 0, 26593
AppSec [baseline] (32.264 ms) : 0, 32264
AppSec [candidate] (34.602 ms) : 0, 34602
Debugger [baseline] (59.803 ms) : 0, 59803
Debugger [candidate] (60.032 ms) : 0, 60032
Remote Config [baseline] (545.231 µs) : 0, 545
Remote Config [candidate] (526.21 µs) : 0, 526
Telemetry [baseline] (7.584 ms) : 0, 7584
Telemetry [candidate] (7.775 ms) : 0, 7775
Flare Poller [baseline] (3.445 ms) : 0, 3445
Flare Poller [candidate] (3.493 ms) : 0, 3493

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.56.0-SNAPSHOT~e750f61cfc, baseline=1.56.0-SNAPSHOT~8d538789b9

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.112 s) : 0, 1111965
Total [baseline] (10.876 s) : 0, 10875886
Agent [candidate] (1.099 s) : 0, 1099107
Total [candidate] (10.728 s) : 0, 10727672
section appsec
Agent [baseline] (1.28 s) : 0, 1279750
Total [baseline] (11.029 s) : 0, 11028787
Agent [candidate] (1.286 s) : 0, 1286097
Total [candidate] (10.925 s) : 0, 10925065
section iast
Agent [baseline] (1.238 s) : 0, 1237872
Total [baseline] (11.184 s) : 0, 11183840
Agent [candidate] (1.236 s) : 0, 1235731
Total [candidate] (11.266 s) : 0, 11265881
section profiling
Agent [baseline] (1.23 s) : 0, 1229602
Total [baseline] (11.034 s) : 0, 11033773
Agent [candidate] (1.236 s) : 0, 1235764
Total [candidate] (11.195 s) : 0, 11195310

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.112 s	-
Agent	appsec	1.28 s	167.784 ms (15.1%)
Agent	iast	1.238 s	125.907 ms (11.3%)
Agent	profiling	1.23 s	117.636 ms (10.6%)
Total	tracing	10.876 s	-
Total	appsec	11.029 s	152.9 ms (1.4%)
Total	iast	11.184 s	307.954 ms (2.8%)
Total	profiling	11.034 s	157.887 ms (1.5%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.099 s	-
Agent	appsec	1.286 s	186.991 ms (17.0%)
Agent	iast	1.236 s	136.624 ms (12.4%)
Agent	profiling	1.236 s	136.658 ms (12.4%)
Total	tracing	10.728 s	-
Total	appsec	10.925 s	197.393 ms (1.8%)
Total	iast	11.266 s	538.209 ms (5.0%)
Total	profiling	11.195 s	467.638 ms (4.4%)

gantt
    title petclinic - break down per module: candidate=1.56.0-SNAPSHOT~e750f61cfc, baseline=1.56.0-SNAPSHOT~8d538789b9

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.465 ms) : 0, 1465
crashtracking [candidate] (1.434 ms) : 0, 1434
BytebuddyAgent [baseline] (713.465 ms) : 0, 713465
BytebuddyAgent [candidate] (705.709 ms) : 0, 705709
GlobalTracer [baseline] (251.199 ms) : 0, 251199
GlobalTracer [candidate] (248.602 ms) : 0, 248602
AppSec [baseline] (32.872 ms) : 0, 32872
AppSec [candidate] (32.155 ms) : 0, 32155
Debugger [baseline] (65.175 ms) : 0, 65175
Debugger [candidate] (63.746 ms) : 0, 63746
Remote Config [baseline] (642.324 µs) : 0, 642
Remote Config [candidate] (629.347 µs) : 0, 629
Telemetry [baseline] (8.309 ms) : 0, 8309
Telemetry [candidate] (8.284 ms) : 0, 8284
Flare Poller [baseline] (3.735 ms) : 0, 3735
Flare Poller [candidate] (3.601 ms) : 0, 3601
section appsec
crashtracking [baseline] (1.458 ms) : 0, 1458
crashtracking [candidate] (1.462 ms) : 0, 1462
BytebuddyAgent [baseline] (730.133 ms) : 0, 730133
BytebuddyAgent [candidate] (733.623 ms) : 0, 733623
GlobalTracer [baseline] (240.484 ms) : 0, 240484
GlobalTracer [candidate] (242.167 ms) : 0, 242167
IAST [baseline] (24.885 ms) : 0, 24885
IAST [candidate] (25.099 ms) : 0, 25099
AppSec [baseline] (174.059 ms) : 0, 174059
AppSec [candidate] (174.553 ms) : 0, 174553
Debugger [baseline] (60.869 ms) : 0, 60869
Debugger [candidate] (60.822 ms) : 0, 60822
Remote Config [baseline] (736.129 µs) : 0, 736
Remote Config [candidate] (676.997 µs) : 0, 677
Telemetry [baseline] (8.253 ms) : 0, 8253
Telemetry [candidate] (8.58 ms) : 0, 8580
Flare Poller [baseline] (3.807 ms) : 0, 3807
Flare Poller [candidate] (3.933 ms) : 0, 3933
section iast
crashtracking [baseline] (1.449 ms) : 0, 1449
crashtracking [candidate] (1.453 ms) : 0, 1453
BytebuddyAgent [baseline] (829.177 ms) : 0, 829177
BytebuddyAgent [candidate] (827.367 ms) : 0, 827367
GlobalTracer [baseline] (238.022 ms) : 0, 238022
GlobalTracer [candidate] (238.291 ms) : 0, 238291
IAST [baseline] (30.089 ms) : 0, 30089
IAST [candidate] (30.86 ms) : 0, 30860
AppSec [baseline] (31.433 ms) : 0, 31433
AppSec [candidate] (30.409 ms) : 0, 30409
Debugger [baseline] (61.047 ms) : 0, 61047
Debugger [candidate] (60.538 ms) : 0, 60538
Remote Config [baseline] (559.008 µs) : 0, 559
Remote Config [candidate] (532.549 µs) : 0, 533
Telemetry [baseline] (7.623 ms) : 0, 7623
Telemetry [candidate] (7.81 ms) : 0, 7810
Flare Poller [baseline] (3.497 ms) : 0, 3497
Flare Poller [candidate] (3.541 ms) : 0, 3541
section profiling
crashtracking [baseline] (1.443 ms) : 0, 1443
crashtracking [candidate] (1.451 ms) : 0, 1451
BytebuddyAgent [baseline] (731.626 ms) : 0, 731626
BytebuddyAgent [candidate] (733.208 ms) : 0, 733208
GlobalTracer [baseline] (222.101 ms) : 0, 222101
GlobalTracer [candidate] (222.345 ms) : 0, 222345
AppSec [baseline] (32.162 ms) : 0, 32162
AppSec [candidate] (32.476 ms) : 0, 32476
Debugger [baseline] (62.955 ms) : 0, 62955
Debugger [candidate] (63.057 ms) : 0, 63057
Remote Config [baseline] (628.872 µs) : 0, 629
Remote Config [candidate] (650.518 µs) : 0, 651
Telemetry [baseline] (7.904 ms) : 0, 7904
Telemetry [candidate] (8.308 ms) : 0, 8308
Flare Poller [baseline] (3.684 ms) : 0, 3684
Flare Poller [candidate] (3.866 ms) : 0, 3866
ProfilingAgent [baseline] (97.375 ms) : 0, 97375
ProfilingAgent [candidate] (100.506 ms) : 0, 100506
Profiling [baseline] (97.965 ms) : 0, 97965
Profiling [candidate] (101.126 ms) : 0, 101126

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	robertomonteromiguel/antithesis_poc_java_tracer
git_commit_date	1763454917	1763631063
git_commit_sha	`8d53878`	`e750f61`
release_version	1.56.0-SNAPSHOT~8d538789b9	1.56.0-SNAPSHOT~e750f61cfc

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1763633633	1763633633
ci_job_id	1246987904	1246987904
ci_pipeline_id	83313668	83313668
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-2-7zck161r 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-2-7zck161r 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 4 performance improvements and 0 performance regressions! Performance is the same for 13 metrics, 19 unstable metrics.

scenario	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p95	Δ mean throughput	candidate mean agg_http_req_duration_p50	candidate mean agg_http_req_duration_p95	candidate mean throughput	baseline mean agg_http_req_duration_p50	baseline mean agg_http_req_duration_p95	baseline mean throughput
scenario:load:insecure-bank:profiling:high_load	better [-162.388µs; -50.860µs] or [-9.339%; -2.925%]	unstable [-1171.037µs; -338.998µs] or [-22.885%; -6.625%]	unstable [-36.069op/s; +482.882op/s] or [-1.734%; +23.217%]	1.632ms	4.362ms	2303.250op/s	1.739ms	5.117ms	2079.844op/s
scenario:load:petclinic:iast:high_load	better [-1.460ms; -0.619ms] or [-7.841%; -3.326%]	better [-2.435ms; -0.973ms] or [-7.975%; -3.186%]	unstable [-11.403op/s; +36.028op/s] or [-4.611%; +14.568%]	17.575ms	28.834ms	259.625op/s	18.614ms	30.539ms	247.312op/s
scenario:load:petclinic:profiling:high_load	better [-1314.593µs; -544.601µs] or [-6.820%; -2.825%]	unsure [-1.946ms; -0.450ms] or [-6.250%; -1.447%]	unstable [-12.248op/s; +32.873op/s] or [-5.134%; +13.780%]	18.347ms	29.940ms	248.875op/s	19.277ms	31.138ms	238.562op/s

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.56.0-SNAPSHOT~e750f61cfc, baseline=1.56.0-SNAPSHOT~8d538789b9
    dateFormat X
    axisFormat %s
section baseline
no_agent (18.314 ms) : 18126, 18502
.   : milestone, 18314,
appsec (18.513 ms) : 18324, 18701
.   : milestone, 18513,
code_origins (17.528 ms) : 17354, 17702
.   : milestone, 17528,
iast (18.868 ms) : 18672, 19064
.   : milestone, 18868,
profiling (19.571 ms) : 19371, 19772
.   : milestone, 19571,
tracing (17.587 ms) : 17411, 17763
.   : milestone, 17587,
section candidate
no_agent (18.092 ms) : 17905, 18279
.   : milestone, 18092,
appsec (18.594 ms) : 18406, 18783
.   : milestone, 18594,
code_origins (17.584 ms) : 17409, 17759
.   : milestone, 17584,
iast (17.972 ms) : 17792, 18153
.   : milestone, 17972,
profiling (18.755 ms) : 18567, 18943
.   : milestone, 18755,
tracing (17.911 ms) : 17732, 18091
.   : milestone, 17911,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.314 ms [18.126 ms, 18.502 ms]	-
appsec	18.513 ms [18.324 ms, 18.701 ms]	198.722 µs (1.1%)
code_origins	17.528 ms [17.354 ms, 17.702 ms]	-786.085 µs (-4.3%)
iast	18.868 ms [18.672 ms, 19.064 ms]	554.001 µs (3.0%)
profiling	19.571 ms [19.371 ms, 19.772 ms]	1.258 ms (6.9%)
tracing	17.587 ms [17.411 ms, 17.763 ms]	-726.586 µs (-4.0%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.092 ms [17.905 ms, 18.279 ms]	-
appsec	18.594 ms [18.406 ms, 18.783 ms]	502.659 µs (2.8%)
code_origins	17.584 ms [17.409 ms, 17.759 ms]	-507.571 µs (-2.8%)
iast	17.972 ms [17.792 ms, 18.153 ms]	-119.471 µs (-0.7%)
profiling	18.755 ms [18.567 ms, 18.943 ms]	663.171 µs (3.7%)
tracing	17.911 ms [17.732 ms, 18.091 ms]	-180.435 µs (-1.0%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.56.0-SNAPSHOT~e750f61cfc, baseline=1.56.0-SNAPSHOT~8d538789b9
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.218 ms) : 1206, 1230
.   : milestone, 1218,
iast (3.201 ms) : 3156, 3246
.   : milestone, 3201,
iast_FULL (5.553 ms) : 5499, 5606
.   : milestone, 5553,
iast_GLOBAL (3.712 ms) : 3645, 3778
.   : milestone, 3712,
profiling (2.176 ms) : 2155, 2196
.   : milestone, 2176,
tracing (1.837 ms) : 1822, 1853
.   : milestone, 1837,
section candidate
no_agent (1.217 ms) : 1204, 1230
.   : milestone, 1217,
iast (3.191 ms) : 3152, 3230
.   : milestone, 3191,
iast_FULL (5.67 ms) : 5614, 5727
.   : milestone, 5670,
iast_GLOBAL (3.693 ms) : 3634, 3752
.   : milestone, 3693,
profiling (1.958 ms) : 1940, 1975
.   : milestone, 1958,
tracing (1.81 ms) : 1795, 1825
.   : milestone, 1810,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.218 ms [1.206 ms, 1.23 ms]	-
iast	3.201 ms [3.156 ms, 3.246 ms]	1.983 ms (162.9%)
iast_FULL	5.553 ms [5.499 ms, 5.606 ms]	4.335 ms (356.0%)
iast_GLOBAL	3.712 ms [3.645 ms, 3.778 ms]	2.494 ms (204.8%)
profiling	2.176 ms [2.155 ms, 2.196 ms]	958.01 µs (78.7%)
tracing	1.837 ms [1.822 ms, 1.853 ms]	619.665 µs (50.9%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.217 ms [1.204 ms, 1.23 ms]	-
iast	3.191 ms [3.152 ms, 3.23 ms]	1.974 ms (162.2%)
iast_FULL	5.67 ms [5.614 ms, 5.727 ms]	4.453 ms (365.9%)
iast_GLOBAL	3.693 ms [3.634 ms, 3.752 ms]	2.476 ms (203.4%)
profiling	1.958 ms [1.94 ms, 1.975 ms]	740.632 µs (60.9%)
tracing	1.81 ms [1.795 ms, 1.825 ms]	593.351 µs (48.8%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	robertomonteromiguel/antithesis_poc_java_tracer
git_commit_date	1763454917	1763631063
git_commit_sha	`8d53878`	`e750f61`
release_version	1.56.0-SNAPSHOT~8d538789b9	1.56.0-SNAPSHOT~e750f61cfc

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1763633198	1763633198
ci_job_id	1246987907	1246987907
ci_pipeline_id	83313668	83313668
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-z32behqd 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-z32behqd 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 1 unstable metrics.

scenario	Δ mean execution_time	candidate mean execution_time	baseline mean execution_time
scenario:dacapo:tomcat:appsec	better [-1.412ms; -1.070ms] or [-38.106%; -28.864%]	2.465ms	3.706ms

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.56.0-SNAPSHOT~e750f61cfc, baseline=1.56.0-SNAPSHOT~8d538789b9
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.822 s) : 14822000, 14822000
.   : milestone, 14822000,
appsec (15.235 s) : 15235000, 15235000
.   : milestone, 15235000,
iast (18.036 s) : 18036000, 18036000
.   : milestone, 18036000,
iast_GLOBAL (17.834 s) : 17834000, 17834000
.   : milestone, 17834000,
profiling (14.935 s) : 14935000, 14935000
.   : milestone, 14935000,
tracing (14.804 s) : 14804000, 14804000
.   : milestone, 14804000,
section candidate
no_agent (15.689 s) : 15689000, 15689000
.   : milestone, 15689000,
appsec (15.366 s) : 15366000, 15366000
.   : milestone, 15366000,
iast (18.25 s) : 18250000, 18250000
.   : milestone, 18250000,
iast_GLOBAL (18.201 s) : 18201000, 18201000
.   : milestone, 18201000,
profiling (14.703 s) : 14703000, 14703000
.   : milestone, 14703000,
tracing (14.53 s) : 14530000, 14530000
.   : milestone, 14530000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.822 s [14.822 s, 14.822 s]	-
appsec	15.235 s [15.235 s, 15.235 s]	413.0 ms (2.8%)
iast	18.036 s [18.036 s, 18.036 s]	3.214 s (21.7%)
iast_GLOBAL	17.834 s [17.834 s, 17.834 s]	3.012 s (20.3%)
profiling	14.935 s [14.935 s, 14.935 s]	113.0 ms (0.8%)
tracing	14.804 s [14.804 s, 14.804 s]	-18.0 ms (-0.1%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.689 s [15.689 s, 15.689 s]	-
appsec	15.366 s [15.366 s, 15.366 s]	-323.0 ms (-2.1%)
iast	18.25 s [18.25 s, 18.25 s]	2.561 s (16.3%)
iast_GLOBAL	18.201 s [18.201 s, 18.201 s]	2.512 s (16.0%)
profiling	14.703 s [14.703 s, 14.703 s]	-986.0 ms (-6.3%)
tracing	14.53 s [14.53 s, 14.53 s]	-1.159 s (-7.4%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.56.0-SNAPSHOT~e750f61cfc, baseline=1.56.0-SNAPSHOT~8d538789b9
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.481 ms) : 1469, 1492
.   : milestone, 1481,
appsec (3.706 ms) : 3487, 3925
.   : milestone, 3706,
iast (2.22 ms) : 2157, 2283
.   : milestone, 2220,
iast_GLOBAL (2.255 ms) : 2191, 2319
.   : milestone, 2255,
profiling (2.486 ms) : 2322, 2649
.   : milestone, 2486,
tracing (2.047 ms) : 1997, 2097
.   : milestone, 2047,
section candidate
no_agent (1.485 ms) : 1474, 1497
.   : milestone, 1485,
appsec (2.465 ms) : 2414, 2516
.   : milestone, 2465,
iast (2.224 ms) : 2160, 2287
.   : milestone, 2224,
iast_GLOBAL (2.265 ms) : 2200, 2329
.   : milestone, 2265,
profiling (2.094 ms) : 2041, 2146
.   : milestone, 2094,
tracing (2.042 ms) : 1992, 2092
.   : milestone, 2042,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.481 ms [1.469 ms, 1.492 ms]	-
appsec	3.706 ms [3.487 ms, 3.925 ms]	2.225 ms (150.2%)
iast	2.22 ms [2.157 ms, 2.283 ms]	738.874 µs (49.9%)
iast_GLOBAL	2.255 ms [2.191 ms, 2.319 ms]	774.32 µs (52.3%)
profiling	2.486 ms [2.322 ms, 2.649 ms]	1.005 ms (67.9%)
tracing	2.047 ms [1.997 ms, 2.097 ms]	566.125 µs (38.2%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.485 ms [1.474 ms, 1.497 ms]	-
appsec	2.465 ms [2.414 ms, 2.516 ms]	979.779 µs (66.0%)
iast	2.224 ms [2.16 ms, 2.287 ms]	738.55 µs (49.7%)
iast_GLOBAL	2.265 ms [2.2 ms, 2.329 ms]	779.363 µs (52.5%)
profiling	2.094 ms [2.041 ms, 2.146 ms]	608.366 µs (41.0%)
tracing	2.042 ms [1.992 ms, 2.092 ms]	557.048 µs (37.5%)

datadog-official · 2025-11-20T10:20:27Z

🎯 Code Coverage
• Patch Coverage: 100.00%
• Total Coverage: 55.74% (-3.94%)

View detailed report

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: e750f61 | Docs | Datadog PR Page | Was this helpful? Give us feedback!}

Antithesis poc

74f1592

robertomonteromiguel added the tag: do not merge Do not merge changes label Nov 18, 2025

robertomonteromiguel added 3 commits November 18, 2025 10:58

Fix compilation?

d7a5e59

fix compilation?

17a3dd7

fix compilation 3

d8dcccb

build with libraries

14f2568

robertomonteromiguel mentioned this pull request Nov 19, 2025

Antithesis DataDog/system-tests#5734

Draft

6 tasks

Java Antithesis log before assertion

e750f61

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Antithesis poc #9993

Antithesis poc #9993

robertomonteromiguel commented Nov 18, 2025

Uh oh!

pr-commenter bot commented Nov 18, 2025 •

edited

Loading

Uh oh!

datadog-official bot commented Nov 20, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Antithesis poc #9993

Are you sure you want to change the base?

Antithesis poc #9993

Conversation

robertomonteromiguel commented Nov 18, 2025

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

pr-commenter bot commented Nov 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

datadog-official bot commented Nov 20, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

pr-commenter bot commented Nov 18, 2025 •

edited

Loading