Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Decouple gradle JDK from compile/test JDK #7506

Draft
wants to merge 20 commits into
base: master
Choose a base branch
from
Draft

Decouple gradle JDK from compile/test JDK #7506

wants to merge 20 commits into from

Conversation

smola
Copy link
Member

@smola smola commented Aug 26, 2024 •
edited by jira bot
Loading

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Jira ticket: APMJAVA-1342

@smola smola added tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels Aug 26, 2024
@smola smola force-pushed the smola/gradle-java11 branch from dc1fa6e to 702851a Compare August 30, 2024 20:49
@pr-commenter
Copy link

pr-commenter bot commented Aug 30, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/gradle-java11
git_commit_date 1737373742 1737373780
git_commit_sha a69247f 920f3ba
release_version 1.46.0-SNAPSHOT~a69247f0aa 1.46.0-SNAPSHOT~920f3ba582
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1737376157 1737376157
ci_job_id 772566135 772566135
ci_pipeline_id 53466526 53466526
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 59 metrics, 4 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.46.0-SNAPSHOT~920f3ba582, baseline=1.46.0-SNAPSHOT~a69247f0aa

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.056 s) : 0, 1055968
Total [baseline] (8.623 s) : 0, 8622580
Agent [candidate] (1.054 s) : 0, 1053669
Total [candidate] (8.652 s) : 0, 8651628
section iast
Agent [baseline] (1.2 s) : 0, 1199904
Total [baseline] (9.207 s) : 0, 9206546
Agent [candidate] (1.197 s) : 0, 1196809
Total [candidate] (9.247 s) : 0, 9247346
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.186 s) : 0, 1185588
Total [baseline] (9.169 s) : 0, 9168916
Agent [candidate] (1.187 s) : 0, 1186961
Total [candidate] (9.19 s) : 0, 9190412
section iast_TELEMETRY_OFF
Agent [baseline] (1.194 s) : 0, 1194267
Total [baseline] (9.226 s) : 0, 9226302
Agent [candidate] (1.178 s) : 0, 1178221
Total [candidate] (9.217 s) : 0, 9216944
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.056 s -
Agent iast 1.2 s 143.936 ms (13.6%)
Agent iast_HARDCODED_SECRET_DISABLED 1.186 s 129.62 ms (12.3%)
Agent iast_TELEMETRY_OFF 1.194 s 138.299 ms (13.1%)
Total tracing 8.623 s -
Total iast 9.207 s 583.966 ms (6.8%)
Total iast_HARDCODED_SECRET_DISABLED 9.169 s 546.336 ms (6.3%)
Total iast_TELEMETRY_OFF 9.226 s 603.723 ms (7.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.054 s -
Agent iast 1.197 s 143.14 ms (13.6%)
Agent iast_HARDCODED_SECRET_DISABLED 1.187 s 133.292 ms (12.7%)
Agent iast_TELEMETRY_OFF 1.178 s 124.552 ms (11.8%)
Total tracing 8.652 s -
Total iast 9.247 s 595.718 ms (6.9%)
Total iast_HARDCODED_SECRET_DISABLED 9.19 s 538.784 ms (6.2%)
Total iast_TELEMETRY_OFF 9.217 s 565.315 ms (6.5%) 
gantt
    title insecure-bank - break down per module: candidate=1.46.0-SNAPSHOT~920f3ba582, baseline=1.46.0-SNAPSHOT~a69247f0aa

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (713.453 ms) : 0, 713453
BytebuddyAgent [candidate] (711.688 ms) : 0, 711688
GlobalTracer [baseline] (255.566 ms) : 0, 255566
GlobalTracer [candidate] (255.944 ms) : 0, 255944
AppSec [baseline] (55.645 ms) : 0, 55645
AppSec [candidate] (57.419 ms) : 0, 57419
Remote Config [baseline] (740.247 µs) : 0, 740
Remote Config [candidate] (739.34 µs) : 0, 739
Telemetry [baseline] (15.653 ms) : 0, 15653
Telemetry [candidate] (12.887 ms) : 0, 12887
section iast
BytebuddyAgent [baseline] (845.475 ms) : 0, 845475
BytebuddyAgent [candidate] (843.985 ms) : 0, 843985
GlobalTracer [baseline] (249.224 ms) : 0, 249224
GlobalTracer [candidate] (247.912 ms) : 0, 247912
AppSec [baseline] (58.566 ms) : 0, 58566
AppSec [candidate] (58.487 ms) : 0, 58487
IAST [baseline] (21.784 ms) : 0, 21784
IAST [candidate] (21.556 ms) : 0, 21556
Remote Config [baseline] (672.58 µs) : 0, 673
Remote Config [candidate] (664.06 µs) : 0, 664
Telemetry [baseline] (8.881 ms) : 0, 8881
Telemetry [candidate] (8.861 ms) : 0, 8861
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (833.828 ms) : 0, 833828
BytebuddyAgent [candidate] (833.305 ms) : 0, 833305
GlobalTracer [baseline] (247.527 ms) : 0, 247527
GlobalTracer [candidate] (248.32 ms) : 0, 248320
AppSec [baseline] (58.093 ms) : 0, 58093
AppSec [candidate] (58.718 ms) : 0, 58718
IAST [baseline] (21.495 ms) : 0, 21495
IAST [candidate] (21.918 ms) : 0, 21918
Remote Config [baseline] (728.307 µs) : 0, 728
Remote Config [candidate] (704.133 µs) : 0, 704
Telemetry [baseline] (8.855 ms) : 0, 8855
Telemetry [candidate] (8.968 ms) : 0, 8968
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (840.639 ms) : 0, 840639
BytebuddyAgent [candidate] (828.454 ms) : 0, 828454
GlobalTracer [baseline] (249.176 ms) : 0, 249176
GlobalTracer [candidate] (246.679 ms) : 0, 246679
AppSec [baseline] (58.541 ms) : 0, 58541
AppSec [candidate] (57.971 ms) : 0, 57971
IAST [baseline] (21.217 ms) : 0, 21217
IAST [candidate] (20.785 ms) : 0, 20785
Remote Config [baseline] (673.587 µs) : 0, 674
Remote Config [candidate] (649.602 µs) : 0, 650
Telemetry [baseline] (8.848 ms) : 0, 8848
Telemetry [candidate] (8.695 ms) : 0, 8695
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.46.0-SNAPSHOT~920f3ba582, baseline=1.46.0-SNAPSHOT~a69247f0aa

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.058 s) : 0, 1058417
Total [baseline] (10.468 s) : 0, 10467624
Agent [candidate] (1.057 s) : 0, 1056748
Total [candidate] (10.491 s) : 0, 10491446
section appsec
Agent [baseline] (1.187 s) : 0, 1187439
Total [baseline] (10.713 s) : 0, 10712966
Agent [candidate] (1.189 s) : 0, 1188618
Total [candidate] (10.708 s) : 0, 10707730
section iast
Agent [baseline] (1.192 s) : 0, 1192192
Total [baseline] (10.974 s) : 0, 10974353
Agent [candidate] (1.186 s) : 0, 1186403
Total [candidate] (10.978 s) : 0, 10978435
section profiling
Agent [baseline] (1.262 s) : 0, 1262299
Total [baseline] (10.838 s) : 0, 10837520
Agent [candidate] (1.255 s) : 0, 1255054
Total [candidate] (10.762 s) : 0, 10762277
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.058 s -
Agent appsec 1.187 s 129.021 ms (12.2%)
Agent iast 1.192 s 133.774 ms (12.6%)
Agent profiling 1.262 s 203.882 ms (19.3%)
Total tracing 10.468 s -
Total appsec 10.713 s 245.341 ms (2.3%)
Total iast 10.974 s 506.729 ms (4.8%)
Total profiling 10.838 s 369.896 ms (3.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.057 s -
Agent appsec 1.189 s 131.87 ms (12.5%)
Agent iast 1.186 s 129.655 ms (12.3%)
Agent profiling 1.255 s 198.307 ms (18.8%)
Total tracing 10.491 s -
Total appsec 10.708 s 216.284 ms (2.1%)
Total iast 10.978 s 486.989 ms (4.6%)
Total profiling 10.762 s 270.83 ms (2.6%) 
gantt
    title petclinic - break down per module: candidate=1.46.0-SNAPSHOT~920f3ba582, baseline=1.46.0-SNAPSHOT~a69247f0aa

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (715.345 ms) : 0, 715345
BytebuddyAgent [candidate] (715.28 ms) : 0, 715280
GlobalTracer [baseline] (256.728 ms) : 0, 256728
GlobalTracer [candidate] (255.688 ms) : 0, 255688
AppSec [baseline] (56.081 ms) : 0, 56081
AppSec [candidate] (55.636 ms) : 0, 55636
Remote Config [baseline] (748.871 µs) : 0, 749
Remote Config [candidate] (730.508 µs) : 0, 731
Telemetry [baseline] (14.547 ms) : 0, 14547
Telemetry [candidate] (14.343 ms) : 0, 14343
section appsec
BytebuddyAgent [baseline] (730.139 ms) : 0, 730139
BytebuddyAgent [candidate] (731.8 ms) : 0, 731800
GlobalTracer [baseline] (252.71 ms) : 0, 252710
GlobalTracer [candidate] (252.698 ms) : 0, 252698
AppSec [baseline] (171.021 ms) : 0, 171021
AppSec [candidate] (170.584 ms) : 0, 170584
Remote Config [baseline] (664.561 µs) : 0, 665
Remote Config [candidate] (660.147 µs) : 0, 660
Telemetry [baseline] (8.216 ms) : 0, 8216
Telemetry [candidate] (8.156 ms) : 0, 8156
IAST [baseline] (19.403 ms) : 0, 19403
IAST [candidate] (19.36 ms) : 0, 19360
section iast
BytebuddyAgent [baseline] (838.561 ms) : 0, 838561
BytebuddyAgent [candidate] (834.644 ms) : 0, 834644
GlobalTracer [baseline] (248.394 ms) : 0, 248394
GlobalTracer [candidate] (247.387 ms) : 0, 247387
AppSec [baseline] (58.662 ms) : 0, 58662
AppSec [candidate] (58.328 ms) : 0, 58328
Remote Config [baseline] (676.564 µs) : 0, 677
Remote Config [candidate] (673.781 µs) : 0, 674
Telemetry [baseline] (8.955 ms) : 0, 8955
Telemetry [candidate] (8.801 ms) : 0, 8801
IAST [baseline] (21.823 ms) : 0, 21823
IAST [candidate] (21.53 ms) : 0, 21530
section profiling
BytebuddyAgent [baseline] (708.168 ms) : 0, 708168
BytebuddyAgent [candidate] (704.946 ms) : 0, 704946
GlobalTracer [baseline] (351.536 ms) : 0, 351536
GlobalTracer [candidate] (350.091 ms) : 0, 350091
AppSec [baseline] (55.202 ms) : 0, 55202
AppSec [candidate] (53.665 ms) : 0, 53665
Remote Config [baseline] (706.603 µs) : 0, 707
Remote Config [candidate] (697.018 µs) : 0, 697
Telemetry [baseline] (8.861 ms) : 0, 8861
Telemetry [candidate] (8.796 ms) : 0, 8796
ProfilingAgent [baseline] (95.69 ms) : 0, 95690
ProfilingAgent [candidate] (94.83 ms) : 0, 94830
Profiling [baseline] (95.714 ms) : 0, 95714
Profiling [candidate] (94.853 ms) : 0, 94853
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-01-20T11:59:12 2025-01-20T12:06:18
git_branch master smola/gradle-java11
git_commit_date 1737373742 1737373780
git_commit_sha a69247f 920f3ba
release_version 1.46.0-SNAPSHOT~a69247f0aa 1.46.0-SNAPSHOT~920f3ba582
start_time 2025-01-20T11:58:57 2025-01-20T12:06:04
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1737375139 1737375139
ci_job_id 772566136 772566136
ci_pipeline_id 53466526 53466526
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 16 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.46.0-SNAPSHOT~920f3ba582, baseline=1.46.0-SNAPSHOT~a69247f0aa
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.365 ms) : 1346, 1385
.   : milestone, 1365,
appsec (1.779 ms) : 1756, 1802
.   : milestone, 1779,
appsec_no_iast (1.765 ms) : 1738, 1793
.   : milestone, 1765,
iast (1.514 ms) : 1489, 1538
.   : milestone, 1514,
profiling (1.512 ms) : 1490, 1535
.   : milestone, 1512,
tracing (1.493 ms) : 1468, 1517
.   : milestone, 1493,
section candidate
no_agent (1.362 ms) : 1343, 1381
.   : milestone, 1362,
appsec (1.762 ms) : 1738, 1786
.   : milestone, 1762,
appsec_no_iast (1.766 ms) : 1742, 1789
.   : milestone, 1766,
iast (1.516 ms) : 1492, 1541
.   : milestone, 1516,
profiling (1.54 ms) : 1515, 1565
.   : milestone, 1540,
tracing (1.499 ms) : 1472, 1526
.   : milestone, 1499,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.365 ms [1.346 ms, 1.385 ms] -
appsec 1.779 ms [1.756 ms, 1.802 ms] 413.593 µs (30.3%)
appsec_no_iast 1.765 ms [1.738 ms, 1.793 ms] 399.87 µs (29.3%)
iast 1.514 ms [1.489 ms, 1.538 ms] 148.136 µs (10.8%)
profiling 1.512 ms [1.49 ms, 1.535 ms] 146.768 µs (10.7%)
tracing 1.493 ms [1.468 ms, 1.517 ms] 127.46 µs (9.3%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.362 ms [1.343 ms, 1.381 ms] -
appsec 1.762 ms [1.738 ms, 1.786 ms] 400.356 µs (29.4%)
appsec_no_iast 1.766 ms [1.742 ms, 1.789 ms] 403.468 µs (29.6%)
iast 1.516 ms [1.492 ms, 1.541 ms] 154.365 µs (11.3%)
profiling 1.54 ms [1.515 ms, 1.565 ms] 177.976 µs (13.1%)
tracing 1.499 ms [1.472 ms, 1.526 ms] 137.135 µs (10.1%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.46.0-SNAPSHOT~920f3ba582, baseline=1.46.0-SNAPSHOT~a69247f0aa
    dateFormat X
    axisFormat %s
section baseline
no_agent (390.545 µs) : 370, 411
.   : milestone, 391,
iast (521.276 µs) : 499, 544
.   : milestone, 521,
iast_FULL (750.928 µs) : 729, 773
.   : milestone, 751,
iast_GLOBAL (561.241 µs) : 539, 583
.   : milestone, 561,
iast_HARDCODED_SECRET_DISABLED (506.705 µs) : 485, 528
.   : milestone, 507,
iast_INACTIVE (467.209 µs) : 445, 489
.   : milestone, 467,
iast_TELEMETRY_OFF (507.608 µs) : 485, 530
.   : milestone, 508,
tracing (460.84 µs) : 440, 482
.   : milestone, 461,
section candidate
no_agent (386.169 µs) : 365, 407
.   : milestone, 386,
iast (511.618 µs) : 490, 534
.   : milestone, 512,
iast_FULL (748.601 µs) : 727, 770
.   : milestone, 749,
iast_GLOBAL (571.613 µs) : 547, 596
.   : milestone, 572,
iast_HARDCODED_SECRET_DISABLED (517.309 µs) : 495, 539
.   : milestone, 517,
iast_INACTIVE (468.176 µs) : 446, 490
.   : milestone, 468,
iast_TELEMETRY_OFF (502.494 µs) : 481, 524
.   : milestone, 502,
tracing (459.428 µs) : 439, 480
.   : milestone, 459,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 390.545 µs [370.373 µs, 410.717 µs] -
iast 521.276 µs [499.05 µs, 543.503 µs] 130.731 µs (33.5%)
iast_FULL 750.928 µs [729.08 µs, 772.776 µs] 360.383 µs (92.3%)
iast_GLOBAL 561.241 µs [539.3 µs, 583.181 µs] 170.695 µs (43.7%)
iast_HARDCODED_SECRET_DISABLED 506.705 µs [485.064 µs, 528.345 µs] 116.16 µs (29.7%)
iast_INACTIVE 467.209 µs [445.422 µs, 488.996 µs] 76.664 µs (19.6%)
iast_TELEMETRY_OFF 507.608 µs [484.986 µs, 530.231 µs] 117.063 µs (30.0%)
tracing 460.84 µs [439.861 µs, 481.819 µs] 70.295 µs (18.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 386.169 µs [365.367 µs, 406.972 µs] -
iast 511.618 µs [489.702 µs, 533.534 µs] 125.449 µs (32.5%)
iast_FULL 748.601 µs [726.703 µs, 770.499 µs] 362.431 µs (93.9%)
iast_GLOBAL 571.613 µs [547.451 µs, 595.775 µs] 185.444 µs (48.0%)
iast_HARDCODED_SECRET_DISABLED 517.309 µs [495.256 µs, 539.362 µs] 131.139 µs (34.0%)
iast_INACTIVE 468.176 µs [446.425 µs, 489.927 µs] 82.007 µs (21.2%)
iast_TELEMETRY_OFF 502.494 µs [480.523 µs, 524.464 µs] 116.324 µs (30.1%)
tracing 459.428 µs [438.813 µs, 480.043 µs] 73.259 µs (19.0%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/gradle-java11
git_commit_date 1737373742 1737373780
git_commit_sha a69247f 920f3ba
release_version 1.46.0-SNAPSHOT~a69247f0aa 1.46.0-SNAPSHOT~920f3ba582
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1737375789 1737375789
ci_job_id 772566137 772566137
ci_pipeline_id 53466526 53466526
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.46.0-SNAPSHOT~920f3ba582, baseline=1.46.0-SNAPSHOT~a69247f0aa
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.466 ms) : 1455, 1478
.   : milestone, 1466,
appsec (2.349 ms) : 2306, 2392
.   : milestone, 2349,
iast (2.093 ms) : 2039, 2147
.   : milestone, 2093,
iast_GLOBAL (2.149 ms) : 2094, 2204
.   : milestone, 2149,
profiling (1.978 ms) : 1933, 2022
.   : milestone, 1978,
tracing (1.936 ms) : 1895, 1978
.   : milestone, 1936,
section candidate
no_agent (1.465 ms) : 1453, 1476
.   : milestone, 1465,
appsec (2.356 ms) : 2312, 2399
.   : milestone, 2356,
iast (2.098 ms) : 2044, 2153
.   : milestone, 2098,
iast_GLOBAL (2.147 ms) : 2092, 2201
.   : milestone, 2147,
profiling (2.453 ms) : 2262, 2643
.   : milestone, 2453,
tracing (1.94 ms) : 1898, 1982
.   : milestone, 1940,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.466 ms [1.455 ms, 1.478 ms] -
appsec 2.349 ms [2.306 ms, 2.392 ms] 882.418 µs (60.2%)
iast 2.093 ms [2.039 ms, 2.147 ms] 626.919 µs (42.8%)
iast_GLOBAL 2.149 ms [2.094 ms, 2.204 ms] 682.695 µs (46.6%)
profiling 1.978 ms [1.933 ms, 2.022 ms] 511.574 µs (34.9%)
tracing 1.936 ms [1.895 ms, 1.978 ms] 469.831 µs (32.0%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.465 ms [1.453 ms, 1.476 ms] -
appsec 2.356 ms [2.312 ms, 2.399 ms] 890.976 µs (60.8%)
iast 2.098 ms [2.044 ms, 2.153 ms] 633.672 µs (43.3%)
iast_GLOBAL 2.147 ms [2.092 ms, 2.201 ms] 682.04 µs (46.6%)
profiling 2.453 ms [2.262 ms, 2.643 ms] 988.058 µs (67.5%)
tracing 1.94 ms [1.898 ms, 1.982 ms] 475.538 µs (32.5%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.46.0-SNAPSHOT~920f3ba582, baseline=1.46.0-SNAPSHOT~a69247f0aa
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.17 s) : 15170000, 15170000
.   : milestone, 15170000,
appsec (15.119 s) : 15119000, 15119000
.   : milestone, 15119000,
iast (18.881 s) : 18881000, 18881000
.   : milestone, 18881000,
iast_GLOBAL (18.286 s) : 18286000, 18286000
.   : milestone, 18286000,
profiling (15.736 s) : 15736000, 15736000
.   : milestone, 15736000,
tracing (14.953 s) : 14953000, 14953000
.   : milestone, 14953000,
section candidate
no_agent (15.491 s) : 15491000, 15491000
.   : milestone, 15491000,
appsec (15.21 s) : 15210000, 15210000
.   : milestone, 15210000,
iast (18.555 s) : 18555000, 18555000
.   : milestone, 18555000,
iast_GLOBAL (17.624 s) : 17624000, 17624000
.   : milestone, 17624000,
profiling (15.652 s) : 15652000, 15652000
.   : milestone, 15652000,
tracing (15.116 s) : 15116000, 15116000
.   : milestone, 15116000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.17 s [15.17 s, 15.17 s] -
appsec 15.119 s [15.119 s, 15.119 s] -51.0 ms (-0.3%)
iast 18.881 s [18.881 s, 18.881 s] 3.711 s (24.5%)
iast_GLOBAL 18.286 s [18.286 s, 18.286 s] 3.116 s (20.5%)
profiling 15.736 s [15.736 s, 15.736 s] 566.0 ms (3.7%)
tracing 14.953 s [14.953 s, 14.953 s] -217.0 ms (-1.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.491 s [15.491 s, 15.491 s] -
appsec 15.21 s [15.21 s, 15.21 s] -281.0 ms (-1.8%)
iast 18.555 s [18.555 s, 18.555 s] 3.064 s (19.8%)
iast_GLOBAL 17.624 s [17.624 s, 17.624 s] 2.133 s (13.8%)
profiling 15.652 s [15.652 s, 15.652 s] 161.0 ms (1.0%)
tracing 15.116 s [15.116 s, 15.116 s] -375.0 ms (-2.4%)

@smola smola force-pushed the smola/gradle-java11 branch from 702851a to d1abf3b Compare September 23, 2024 11:18
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 1, 2025

This pull request has been marked as stale because it has not had activity over the past quarter. It will be closed in 7 days if no further activity occurs. Feel free to reopen the PR if you are still working on it.

@github-actions github-actions bot added the tag: stale Stale pull requests label Jan 1, 2025
@smola smola force-pushed the smola/gradle-java11 branch 2 times, most recently from 0e79c09 to 0a43968 Compare January 15, 2025 15:19
@pr-commenter
Copy link

pr-commenter bot commented Jan 15, 2025 •
edited
Loading

Debugger benchmarks

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
ci_job_date 1737357243 1737357619
end_time 2025-01-20T07:15:19 2025-01-20T07:21:36
git_branch master smola/gradle-java11
git_commit_sha 0b239ae 352736e
start_time 2025-01-20T07:14:04 2025-01-20T07:20:20
See matching parameters
Baseline Candidate
ci_job_id 772223798 772223798
ci_pipeline_id 53445931 53445931
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date 1737222884 1737222884

Summary

Found 0 performance improvements and 6 performance regressions! Performance is the same for 3 metrics, 6 unstable metrics.

scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:basic worse
[+3.576µs; +11.815µs] or [+1.324%; +4.374%]		 same same unstable
[-23.734µs; +39.109µs] or [-4.673%; +7.700%]		 unstable
[-239.733op/s; +111.528op/s] or [-9.350%; +4.350%]
scenario:loop worse
[+442.270µs; +522.596µs] or [+4.356%; +5.147%]		 worse
[+449.109µs; +550.782µs] or [+4.353%; +5.338%]		 worse
[+410.327µs; +544.125µs] or [+3.935%; +5.218%]		 worse
[+195.678µs; +703.388µs] or [+1.812%; +6.512%]		 worse
[-5.491op/s; -3.248op/s] or [-5.738%; -3.394%]
See unchanged results
scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:noprobe unstable
[-26.397µs; +21.584µs] or [-9.744%; +7.968%]		 unstable
[-39.342µs; +33.189µs] or [-12.663%; +10.682%]		 unstable
[-52.143µs; +42.935µs] or [-16.038%; +13.206%]		 unstable
[-94.545µs; +97.240µs] or [-16.186%; +16.647%]		 same
Request duration reports for reports 
gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (310.695 µs) : 270, 352
.   : milestone, 311,
basic (301.977 µs) : 293, 310
.   : milestone, 302,
loop (10.317 ms) : 10257, 10378
.   : milestone, 10317,
section candidate
noprobe (307.618 µs) : 281, 334
.   : milestone, 308,
basic (309.887 µs) : 301, 319
.   : milestone, 310,
loop (10.817 ms) : 10788, 10847
.   : milestone, 10817,
Loading
  • baseline results
Scenario Request median duration [CI 0.99]
noprobe 310.695 µs [269.778 µs, 351.611 µs]
basic 301.977 µs [293.457 µs, 310.498 µs]
loop 10.317 ms [10.257 ms, 10.378 ms]
  • candidate results
Scenario Request median duration [CI 0.99]
noprobe 307.618 µs [280.843 µs, 334.393 µs]
basic 309.887 µs [301.101 µs, 318.674 µs]
loop 10.817 ms [10.788 ms, 10.847 ms]

@smola smola force-pushed the smola/gradle-java11 branch 3 times, most recently from 22e3221 to 352736e Compare January 20, 2025 06:59
@pr-commenter
Copy link

pr-commenter bot commented Jan 20, 2025 •
edited
Loading

Kafka / producer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/gradle-java11
git_commit_date 1737373742 1737373780
git_commit_sha a69247f 920f3ba
See matching parameters
Baseline Candidate
ci_job_date 1737374857 1737374857
ci_job_id 772566139 772566139
ci_pipeline_id 53466526 53466526
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.25 11.0.25
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.25+9-post-Ubuntu-1ubuntu122.04 11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce same

@pr-commenter
Copy link

pr-commenter bot commented Jan 20, 2025 •
edited
Loading

Kafka / consumer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/gradle-java11
git_commit_date 1737373742 1737373780
git_commit_sha a69247f 920f3ba
See matching parameters
Baseline Candidate
ci_job_date 1737374919 1737374919
ci_job_id 772566140 772566140
ci_pipeline_id 53466526 53466526
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.25 11.0.25
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.25+9-post-Ubuntu-1ubuntu122.04 11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume same

@smola smola removed the tag: stale Stale pull requests label Jan 20, 2025
@smola smola force-pushed the smola/gradle-java11 branch from 352736e to 920f3ba Compare January 20, 2025 11:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
comp: tooling Build & Tooling tag: no release notes Changes to exclude from release notes
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@smola