[MLOB] add LLM obs configs #8076

gary-huang · 2024-12-10T20:57:34Z

What Does This Do

adds llm observability related configs and some start up behavior that are related to the enablement of llm observability

if dd.llmobs.enabled is set to true and there is no ml app defined - throw error
if dd.llmobs.agentless.enabled is set to true and there is no API key supplied - throw error

java system property success cases
agent mode configuration

$ java -javaagent:/Users/gary.huang/dd/dd-trace-java/dd-java-agent/build/libs/dd-java-agent-1.44.0-SNAPSHOT.jar -Ddd.profiling.enabled=true -XX:FlightRecorderOptions=stackdepth=256 -Ddd.logs.injection=true -Ddd.service=gary-test -Ddd.env=staging -Ddd.trace.debug=true -Ddd.version=1.0 -Ddd.llmobs.enabled=true -Ddd.llmobs.ml.app=gary-test -jar build/libs/rest-service-0.0.1-SNAPSHOT.jar 
...
[dd.trace 2024-12-18 17:05:37:610 -0500] [main] DEBUG datadog.trace.api.Config - Attempting to enable LLM Observability
[dd.trace 2024-12-18 17:05:37:613 -0500] [main] DEBUG datadog.trace.api.Config - LLM Observability enabled for ML app gary-test, agentless mode false
...

agentless mode configuration with API key env var

$ DD_API_KEY=${DDOG_API_KEY} java -javaagent:/Users/gary.huang/dd/dd-trace-java/dd-java-agent/build/libs/dd-java-agent-1.44.0-SNAPSHOT.jar -Ddd.profiling.enabled=true -XX:FlightRecorderOptions=stackdepth=256 -Ddd.logs.injection=true -Ddd.service=gary-test -Ddd.env=staging -Ddd.trace.debug=true -Ddd.version=1.0 -Ddd.llmobs.enabled=true -Ddd.llmobs.ml.app=gary-test -Ddd.llmobs.agentless.enabled=true -jar build/libs/rest-service-0.0.1-SNAPSHOT.jar
...
[dd.trace 2024-12-19 12:10:37:358 -0500] [main] DEBUG datadog.trace.api.Config - Attempting to enable LLM Observability
[dd.trace 2024-12-19 12:10:37:361 -0500] [main] DEBUG datadog.trace.api.Config - LLM Observability enabled for ML app gary-test, agentless mode true
...

env variable success cases
agent mode configuration

$ DD_LLMOBS_ML_APP=gary-test-env java -javaagent:/Users/gary.huang/dd/dd-trace-java/dd-java-agent/build/libs/dd-java-agent-1.44.0-SNAPSHOT.jar -Ddd.profiling.enabled=true -XX:FlightRecorderOptions=stackdepth=256 -Ddd.logs.injection=true -Ddd.service=gary-test -Ddd.env=staging -Ddd.trace.debug=true -Ddd.version=1.0 -jar build/libs/rest-service-0.0.1-SNAPSHOT.jar
...
[dd.trace 2024-12-19 13:13:14:104 -0500] [main] DEBUG datadog.trace.api.Config - Attempting to enable LLM Observability
[dd.trace 2024-12-19 13:13:14:108 -0500] [main] DEBUG datadog.trace.api.Config - LLM Observability enabled for ML app gary-test-env, agentless mode false
...

agentless mode configuration

$ DD_LLMOBS_ENABLED=true DD_LLMOBS_ML_APP=gary-test-env DD_LLMOBS_AGENTLESS_ENABLED=true DD_API_KEY=${DDOG_API_KEY} java -javaagent:/Users/gary.huang/dd/dd-trace-java/dd-java-agent/build/libs/dd-java-agent-1.44.0-SNAPSHOT.jar -Ddd.profiling.enabled=true -XX:FlightRecorderOptions=stackdepth=256 -Ddd.logs.injection=true -Ddd.service=gary-test -Ddd.env=staging -Ddd.trace.debug=true -Ddd.version=1.0 -jar build/libs/rest-service-0.0.1-SNAPSHOT.jar
...
[dd.trace 2024-12-19 18:32:45:378 -0500] [main] DEBUG datadog.trace.api.Config - Attempting to enable LLM Observability
[dd.trace 2024-12-19 18:32:45:381 -0500] [main] DEBUG datadog.trace.api.Config - LLM Observability enabled for ML app gary-test-env, agentless mode true
...

Motivation

Additional Notes

tested with a sample spring app
the following stack satisfies 1

$ java -javaagent:/Users/gary.huang/dd/dd-trace-java/dd-java-agent/build/libs/dd-java-agent-1.44.0-SNAPSHOT.jar -Ddd.profiling.enabled=true -XX:FlightRecorderOptions=stackdepth=256 -Ddd.logs.injection=true -Ddd.service=gary-test -Ddd.env=staging -Ddd.trace.debug=true -Ddd.version=1.0 -Ddd.llmobs.enabled=true -jar build/libs/rest-service-0.0.1-SNAPSHOT.jar
...
[dd.trace 2024-12-18 16:51:33:865 -0500] [main] ERROR datadog.trace.bootstrap.Agent - Throwable thrown while installing the Datadog Agent
java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
...
	at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:491)
	at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:503)
Caused by: java.lang.NoClassDefFoundError: Could not initialize class datadog.trace.api.Config
	at datadog.trace.instrumentation.log4j2.LoggerConfigInstrumentation.isApplicable(LoggerConfigInstrumentation.java:26)
	at datadog.trace.agent.tooling.AgentInstaller.installBytebuddyAgent(AgentInstaller.java:203)
	at datadog.trace.agent.tooling.AgentInstaller.installBytebuddyAgent(AgentInstaller.java:79)
	... 19 more
Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.IllegalArgumentException: Attempt to enable LLM Observability without ML app defined.Please ensure that the name of the ML app is provided through properties or env variable [in thread "main"]
	at datadog.trace.api.Config.<init>(Config.java:1767)
...
	... 13 more
[dd.trace 2024-12-18 16:51:33:873 -0500] [main] ERROR datadog.trace.bootstrap.Agent - Throwable thrown while installing the Datadog Tracer
...

the follow stack satisfies 2

$ java -javaagent:/Users/gary.huang/dd/dd-trace-java/dd-java-agent/build/libs/dd-java-agent-1.44.0-SNAPSHOT.jar -Ddd.profiling.enabled=true -XX:FlightRecorderOptions=stackdepth=256 -Ddd.logs.injection=true -Ddd.service=gary-test -Ddd.env=staging -Ddd.trace.debug=true -Ddd.version=1.0 -Ddd.llmobs.enabled=true -Ddd.llmobs.ml.app=gary-test -Ddd.llmobs.agentless.enabled=true -jar build/libs/rest-service-0.0.1-SNAPSHOT.jar 
...
[dd.trace 2024-12-18 17:07:49:861 -0500] [main] ERROR datadog.trace.bootstrap.Agent - Throwable thrown while installing the Datadog Agent
java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
...
	at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:491)
	at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:503)
Caused by: java.lang.NoClassDefFoundError: Could not initialize class datadog.trace.api.Config
	at datadog.trace.instrumentation.log4j2.LoggerConfigInstrumentation.isApplicable(LoggerConfigInstrumentation.java:26)
	at datadog.trace.agent.tooling.AgentInstaller.installBytebuddyAgent(AgentInstaller.java:203)
	at datadog.trace.agent.tooling.AgentInstaller.installBytebuddyAgent(AgentInstaller.java:79)
	... 19 more
Caused by: java.lang.ExceptionInInitializerError: Exception datadog.trace.util.throwable.FatalAgentMisconfigurationError: Attempt to start LLM Observability in Agentless mode without API key. Please ensure that either an API key is configured, or the tracer is set up to work with the Agent [in thread "main"]
	at datadog.trace.api.Config.<init>(Config.java:1773)
	at datadog.trace.api.Config.<clinit>(Config.java:4082)
...
	at datadog.trace.bootstrap.Agent.start(Agent.java:279)
	... 13 more
[dd.trace 2024-12-18 17:07:49:864 -0500] [main] ERROR datadog.trace.bootstrap.Agent - Throwable thrown while installing the Datadog Tracer
java.lang.NoClassDefFoundError: Could not initialize class datadog.trace.api.Config
	at datadog.trace.bootstrap.Agent.createProfilingContextIntegration(Agent.java:980)
	at datadog.trace.bootstrap.Agent.installDatadogTracer(Agent.java:647)
...

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Squash your commits prior merging or merge using GitHub's Squash and merge
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

pr-commenter · 2024-12-10T22:11:17Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	gary/add-llmobs-configs
git_commit_date	1734694675	1734712115
git_commit_sha	`a3e9bda`	`a8108c6`
release_version	1.45.0-SNAPSHOT~a3e9bda406	1.45.0-SNAPSHOT~a8108c61a2

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1734714643	1734714643
ci_job_id	747239006	747239006
ci_pipeline_id	51665654	51665654
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 57 metrics, 6 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.45.0-SNAPSHOT~a8108c61a2, baseline=1.45.0-SNAPSHOT~a3e9bda406

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.052 s) : 0, 1052178
Total [baseline] (8.592 s) : 0, 8591785
Agent [candidate] (1.054 s) : 0, 1053561
Total [candidate] (8.61 s) : 0, 8610141
section iast
Agent [baseline] (1.175 s) : 0, 1174989
Total [baseline] (9.201 s) : 0, 9201019
Agent [candidate] (1.195 s) : 0, 1195111
Total [candidate] (9.25 s) : 0, 9249571
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.176 s) : 0, 1175769
Total [baseline] (9.18 s) : 0, 9179874
Agent [candidate] (1.18 s) : 0, 1179601
Total [candidate] (9.243 s) : 0, 9242588
section iast_TELEMETRY_OFF
Agent [baseline] (1.172 s) : 0, 1172257
Total [baseline] (9.172 s) : 0, 9172262
Agent [candidate] (1.175 s) : 0, 1175367
Total [candidate] (9.228 s) : 0, 9227922

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.052 s	-
Agent	iast	1.175 s	122.811 ms (11.7%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.176 s	123.591 ms (11.7%)
Agent	iast_TELEMETRY_OFF	1.172 s	120.079 ms (11.4%)
Total	tracing	8.592 s	-
Total	iast	9.201 s	609.234 ms (7.1%)
Total	iast_HARDCODED_SECRET_DISABLED	9.18 s	588.09 ms (6.8%)
Total	iast_TELEMETRY_OFF	9.172 s	580.477 ms (6.8%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.054 s	-
Agent	iast	1.195 s	141.55 ms (13.4%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.18 s	126.04 ms (12.0%)
Agent	iast_TELEMETRY_OFF	1.175 s	121.806 ms (11.6%)
Total	tracing	8.61 s	-
Total	iast	9.25 s	639.429 ms (7.4%)
Total	iast_HARDCODED_SECRET_DISABLED	9.243 s	632.447 ms (7.3%)
Total	iast_TELEMETRY_OFF	9.228 s	617.781 ms (7.2%)

gantt
    title insecure-bank - break down per module: candidate=1.45.0-SNAPSHOT~a8108c61a2, baseline=1.45.0-SNAPSHOT~a3e9bda406

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (710.956 ms) : 0, 710956
BytebuddyAgent [candidate] (712.065 ms) : 0, 712065
GlobalTracer [baseline] (255.926 ms) : 0, 255926
GlobalTracer [candidate] (256.173 ms) : 0, 256173
AppSec [baseline] (57.793 ms) : 0, 57793
AppSec [candidate] (57.819 ms) : 0, 57819
Remote Config [baseline] (679.188 µs) : 0, 679
Remote Config [candidate] (695.282 µs) : 0, 695
Telemetry [baseline] (11.839 ms) : 0, 11839
Telemetry [candidate] (11.857 ms) : 0, 11857
section iast
BytebuddyAgent [baseline] (826.472 ms) : 0, 826472
BytebuddyAgent [candidate] (841.011 ms) : 0, 841011
GlobalTracer [baseline] (245.702 ms) : 0, 245702
GlobalTracer [candidate] (249.238 ms) : 0, 249238
AppSec [baseline] (57.808 ms) : 0, 57808
AppSec [candidate] (58.645 ms) : 0, 58645
IAST [baseline] (21.024 ms) : 0, 21024
IAST [candidate] (21.751 ms) : 0, 21751
Remote Config [baseline] (668.477 µs) : 0, 668
Remote Config [candidate] (689.404 µs) : 0, 689
Telemetry [baseline] (8.424 ms) : 0, 8424
Telemetry [candidate] (8.636 ms) : 0, 8636
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (826.44 ms) : 0, 826440
BytebuddyAgent [candidate] (829.249 ms) : 0, 829249
GlobalTracer [baseline] (246.263 ms) : 0, 246263
GlobalTracer [candidate] (247.251 ms) : 0, 247251
AppSec [baseline] (57.806 ms) : 0, 57806
AppSec [candidate] (57.806 ms) : 0, 57806
IAST [baseline] (21.193 ms) : 0, 21193
IAST [candidate] (21.173 ms) : 0, 21173
Remote Config [baseline] (650.224 µs) : 0, 650
Remote Config [candidate] (667.21 µs) : 0, 667
Telemetry [baseline] (8.499 ms) : 0, 8499
Telemetry [candidate] (8.483 ms) : 0, 8483
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (823.871 ms) : 0, 823871
BytebuddyAgent [candidate] (826.1 ms) : 0, 826100
GlobalTracer [baseline] (246.475 ms) : 0, 246475
GlobalTracer [candidate] (246.987 ms) : 0, 246987
AppSec [baseline] (57.293 ms) : 0, 57293
AppSec [candidate] (57.571 ms) : 0, 57571
IAST [baseline] (20.743 ms) : 0, 20743
IAST [candidate] (20.802 ms) : 0, 20802
Remote Config [baseline] (646.287 µs) : 0, 646
Remote Config [candidate] (660.978 µs) : 0, 661
Telemetry [baseline] (8.303 ms) : 0, 8303
Telemetry [candidate] (8.306 ms) : 0, 8306

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.45.0-SNAPSHOT~a8108c61a2, baseline=1.45.0-SNAPSHOT~a3e9bda406

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.052 s) : 0, 1051968
Total [baseline] (10.392 s) : 0, 10392238
Agent [candidate] (1.059 s) : 0, 1059170
Total [candidate] (10.443 s) : 0, 10442542
section appsec
Agent [baseline] (1.193 s) : 0, 1193435
Total [baseline] (10.793 s) : 0, 10793360
Agent [candidate] (1.193 s) : 0, 1192771
Total [candidate] (10.756 s) : 0, 10756392
section iast
Agent [baseline] (1.184 s) : 0, 1184461
Total [baseline] (11.015 s) : 0, 11014811
Agent [candidate] (1.177 s) : 0, 1177484
Total [candidate] (10.981 s) : 0, 10981441
section profiling
Agent [baseline] (1.271 s) : 0, 1271102
Total [baseline] (10.798 s) : 0, 10797619
Agent [candidate] (1.28 s) : 0, 1279710
Total [candidate] (10.904 s) : 0, 10904160

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.052 s	-
Agent	appsec	1.193 s	141.467 ms (13.4%)
Agent	iast	1.184 s	132.493 ms (12.6%)
Agent	profiling	1.271 s	219.134 ms (20.8%)
Total	tracing	10.392 s	-
Total	appsec	10.793 s	401.122 ms (3.9%)
Total	iast	11.015 s	622.573 ms (6.0%)
Total	profiling	10.798 s	405.381 ms (3.9%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.059 s	-
Agent	appsec	1.193 s	133.601 ms (12.6%)
Agent	iast	1.177 s	118.314 ms (11.2%)
Agent	profiling	1.28 s	220.541 ms (20.8%)
Total	tracing	10.443 s	-
Total	appsec	10.756 s	313.85 ms (3.0%)
Total	iast	10.981 s	538.899 ms (5.2%)
Total	profiling	10.904 s	461.618 ms (4.4%)

gantt
    title petclinic - break down per module: candidate=1.45.0-SNAPSHOT~a8108c61a2, baseline=1.45.0-SNAPSHOT~a3e9bda406

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (710.453 ms) : 0, 710453
BytebuddyAgent [candidate] (716.448 ms) : 0, 716448
GlobalTracer [baseline] (255.606 ms) : 0, 255606
GlobalTracer [candidate] (257.681 ms) : 0, 257681
AppSec [baseline] (57.78 ms) : 0, 57780
AppSec [candidate] (58.868 ms) : 0, 58868
Remote Config [baseline] (689.974 µs) : 0, 690
Remote Config [candidate] (681.278 µs) : 0, 681
Telemetry [baseline] (12.557 ms) : 0, 12557
Telemetry [candidate] (10.446 ms) : 0, 10446
section appsec
BytebuddyAgent [baseline] (733.481 ms) : 0, 733481
BytebuddyAgent [candidate] (733.059 ms) : 0, 733059
GlobalTracer [baseline] (254.587 ms) : 0, 254587
GlobalTracer [candidate] (254.29 ms) : 0, 254290
AppSec [baseline] (171.925 ms) : 0, 171925
AppSec [candidate] (171.583 ms) : 0, 171583
IAST [baseline] (19.437 ms) : 0, 19437
IAST [candidate] (19.453 ms) : 0, 19453
Remote Config [baseline] (660.121 µs) : 0, 660
Remote Config [candidate] (667.143 µs) : 0, 667
Telemetry [baseline] (7.981 ms) : 0, 7981
Telemetry [candidate] (8.383 ms) : 0, 8383
section iast
BytebuddyAgent [baseline] (833.75 ms) : 0, 833750
BytebuddyAgent [candidate] (828.516 ms) : 0, 828516
GlobalTracer [baseline] (247.358 ms) : 0, 247358
GlobalTracer [candidate] (246.013 ms) : 0, 246013
AppSec [baseline] (57.77 ms) : 0, 57770
AppSec [candidate] (57.836 ms) : 0, 57836
IAST [baseline] (21.269 ms) : 0, 21269
IAST [candidate] (21.105 ms) : 0, 21105
Remote Config [baseline] (664.603 µs) : 0, 665
Remote Config [candidate] (647.119 µs) : 0, 647
Telemetry [baseline] (8.574 ms) : 0, 8574
Telemetry [candidate] (8.398 ms) : 0, 8398
section profiling
BytebuddyAgent [baseline] (700.204 ms) : 0, 700204
BytebuddyAgent [candidate] (704.897 ms) : 0, 704897
GlobalTracer [baseline] (371.167 ms) : 0, 371167
GlobalTracer [candidate] (373.499 ms) : 0, 373499
AppSec [baseline] (54.029 ms) : 0, 54029
AppSec [candidate] (54.218 ms) : 0, 54218
Remote Config [baseline] (645.378 µs) : 0, 645
Remote Config [candidate] (659.068 µs) : 0, 659
Telemetry [baseline] (7.768 ms) : 0, 7768
Telemetry [candidate] (7.913 ms) : 0, 7913
ProfilingAgent [baseline] (95.574 ms) : 0, 95574
ProfilingAgent [candidate] (96.391 ms) : 0, 96391
Profiling [baseline] (95.598 ms) : 0, 95598
Profiling [candidate] (96.415 ms) : 0, 96415

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2024-12-20T16:40:58	2024-12-20T16:47:59
git_branch	master	gary/add-llmobs-configs
git_commit_date	1734694675	1734712115
git_commit_sha	`a3e9bda`	`a8108c6`
release_version	1.45.0-SNAPSHOT~a3e9bda406	1.45.0-SNAPSHOT~a8108c61a2
start_time	2024-12-20T16:40:45	2024-12-20T16:47:46

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1734713634	1734713634
ci_job_id	747239007	747239007
ci_pipeline_id	51665654	51665654
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 13 metrics, 15 unstable metrics.

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.45.0-SNAPSHOT~a8108c61a2, baseline=1.45.0-SNAPSHOT~a3e9bda406
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.365 ms) : 1346, 1385
.   : milestone, 1365,
appsec (1.76 ms) : 1734, 1786
.   : milestone, 1760,
appsec_no_iast (1.765 ms) : 1741, 1789
.   : milestone, 1765,
iast (1.52 ms) : 1497, 1543
.   : milestone, 1520,
profiling (1.56 ms) : 1536, 1585
.   : milestone, 1560,
tracing (1.501 ms) : 1477, 1525
.   : milestone, 1501,
section candidate
no_agent (1.374 ms) : 1355, 1393
.   : milestone, 1374,
appsec (1.77 ms) : 1746, 1794
.   : milestone, 1770,
appsec_no_iast (1.756 ms) : 1732, 1781
.   : milestone, 1756,
iast (1.522 ms) : 1499, 1544
.   : milestone, 1522,
profiling (1.517 ms) : 1494, 1540
.   : milestone, 1517,
tracing (1.494 ms) : 1465, 1522
.   : milestone, 1494,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.365 ms [1.346 ms, 1.385 ms]	-
appsec	1.76 ms [1.734 ms, 1.786 ms]	394.834 µs (28.9%)
appsec_no_iast	1.765 ms [1.741 ms, 1.789 ms]	400.184 µs (29.3%)
iast	1.52 ms [1.497 ms, 1.543 ms]	154.821 µs (11.3%)
profiling	1.56 ms [1.536 ms, 1.585 ms]	194.994 µs (14.3%)
tracing	1.501 ms [1.477 ms, 1.525 ms]	135.8 µs (9.9%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.374 ms [1.355 ms, 1.393 ms]	-
appsec	1.77 ms [1.746 ms, 1.794 ms]	395.584 µs (28.8%)
appsec_no_iast	1.756 ms [1.732 ms, 1.781 ms]	382.363 µs (27.8%)
iast	1.522 ms [1.499 ms, 1.544 ms]	147.529 µs (10.7%)
profiling	1.517 ms [1.494 ms, 1.54 ms]	143.285 µs (10.4%)
tracing	1.494 ms [1.465 ms, 1.522 ms]	119.412 µs (8.7%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.45.0-SNAPSHOT~a8108c61a2, baseline=1.45.0-SNAPSHOT~a3e9bda406
    dateFormat X
    axisFormat %s
section baseline
no_agent (380.912 µs) : 360, 401
.   : milestone, 381,
iast (501.216 µs) : 480, 523
.   : milestone, 501,
iast_FULL (658.398 µs) : 637, 680
.   : milestone, 658,
iast_GLOBAL (522.873 µs) : 501, 544
.   : milestone, 523,
iast_HARDCODED_SECRET_DISABLED (499.192 µs) : 477, 521
.   : milestone, 499,
iast_INACTIVE (461.362 µs) : 440, 483
.   : milestone, 461,
iast_TELEMETRY_OFF (488.212 µs) : 466, 511
.   : milestone, 488,
tracing (457.463 µs) : 436, 479
.   : milestone, 457,
section candidate
no_agent (381.705 µs) : 361, 402
.   : milestone, 382,
iast (500.516 µs) : 478, 523
.   : milestone, 501,
iast_FULL (661.447 µs) : 640, 683
.   : milestone, 661,
iast_GLOBAL (520.915 µs) : 499, 543
.   : milestone, 521,
iast_HARDCODED_SECRET_DISABLED (506.074 µs) : 484, 528
.   : milestone, 506,
iast_INACTIVE (456.382 µs) : 435, 477
.   : milestone, 456,
iast_TELEMETRY_OFF (491.042 µs) : 469, 513
.   : milestone, 491,
tracing (450.016 µs) : 430, 470
.   : milestone, 450,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	380.912 µs [360.383 µs, 401.441 µs]	-
iast	501.216 µs [479.539 µs, 522.893 µs]	120.304 µs (31.6%)
iast_FULL	658.398 µs [636.722 µs, 680.073 µs]	277.485 µs (72.8%)
iast_GLOBAL	522.873 µs [501.475 µs, 544.271 µs]	141.961 µs (37.3%)
iast_HARDCODED_SECRET_DISABLED	499.192 µs [477.355 µs, 521.03 µs]	118.28 µs (31.1%)
iast_INACTIVE	461.362 µs [439.792 µs, 482.932 µs]	80.45 µs (21.1%)
iast_TELEMETRY_OFF	488.212 µs [465.911 µs, 510.514 µs]	107.3 µs (28.2%)
tracing	457.463 µs [436.171 µs, 478.754 µs]	76.551 µs (20.1%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	381.705 µs [361.345 µs, 402.064 µs]	-
iast	500.516 µs [478.281 µs, 522.751 µs]	118.811 µs (31.1%)
iast_FULL	661.447 µs [639.644 µs, 683.25 µs]	279.742 µs (73.3%)
iast_GLOBAL	520.915 µs [499.151 µs, 542.679 µs]	139.21 µs (36.5%)
iast_HARDCODED_SECRET_DISABLED	506.074 µs [484.197 µs, 527.95 µs]	124.369 µs (32.6%)
iast_INACTIVE	456.382 µs [435.487 µs, 477.277 µs]	74.677 µs (19.6%)
iast_TELEMETRY_OFF	491.042 µs [468.901 µs, 513.182 µs]	109.337 µs (28.6%)
tracing	450.016 µs [429.63 µs, 470.403 µs]	68.312 µs (17.9%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	gary/add-llmobs-configs
git_commit_date	1734694675	1734712115
git_commit_sha	`a3e9bda`	`a8108c6`
release_version	1.45.0-SNAPSHOT~a3e9bda406	1.45.0-SNAPSHOT~a8108c61a2

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1734714120	1734714120
ci_job_id	747239008	747239008
ci_pipeline_id	51665654	51665654
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.45.0-SNAPSHOT~a8108c61a2, baseline=1.45.0-SNAPSHOT~a3e9bda406
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.61 s) : 15610000, 15610000
.   : milestone, 15610000,
appsec (15.077 s) : 15077000, 15077000
.   : milestone, 15077000,
iast (18.921 s) : 18921000, 18921000
.   : milestone, 18921000,
iast_GLOBAL (17.903 s) : 17903000, 17903000
.   : milestone, 17903000,
profiling (15.174 s) : 15174000, 15174000
.   : milestone, 15174000,
tracing (14.881 s) : 14881000, 14881000
.   : milestone, 14881000,
section candidate
no_agent (14.96 s) : 14960000, 14960000
.   : milestone, 14960000,
appsec (15.201 s) : 15201000, 15201000
.   : milestone, 15201000,
iast (18.348 s) : 18348000, 18348000
.   : milestone, 18348000,
iast_GLOBAL (18.204 s) : 18204000, 18204000
.   : milestone, 18204000,
profiling (15.15 s) : 15150000, 15150000
.   : milestone, 15150000,
tracing (14.911 s) : 14911000, 14911000
.   : milestone, 14911000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.61 s [15.61 s, 15.61 s]	-
appsec	15.077 s [15.077 s, 15.077 s]	-533.0 ms (-3.4%)
iast	18.921 s [18.921 s, 18.921 s]	3.311 s (21.2%)
iast_GLOBAL	17.903 s [17.903 s, 17.903 s]	2.293 s (14.7%)
profiling	15.174 s [15.174 s, 15.174 s]	-436.0 ms (-2.8%)
tracing	14.881 s [14.881 s, 14.881 s]	-729.0 ms (-4.7%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.96 s [14.96 s, 14.96 s]	-
appsec	15.201 s [15.201 s, 15.201 s]	241.0 ms (1.6%)
iast	18.348 s [18.348 s, 18.348 s]	3.388 s (22.6%)
iast_GLOBAL	18.204 s [18.204 s, 18.204 s]	3.244 s (21.7%)
profiling	15.15 s [15.15 s, 15.15 s]	190.0 ms (1.3%)
tracing	14.911 s [14.911 s, 14.911 s]	-49.0 ms (-0.3%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.45.0-SNAPSHOT~a8108c61a2, baseline=1.45.0-SNAPSHOT~a3e9bda406
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.461 ms) : 1450, 1472
.   : milestone, 1461,
appsec (2.321 ms) : 2280, 2363
.   : milestone, 2321,
iast (2.072 ms) : 2019, 2124
.   : milestone, 2072,
iast_GLOBAL (2.116 ms) : 2063, 2169
.   : milestone, 2116,
profiling (1.944 ms) : 1902, 1986
.   : milestone, 1944,
tracing (1.927 ms) : 1887, 1968
.   : milestone, 1927,
section candidate
no_agent (1.46 ms) : 1448, 1471
.   : milestone, 1460,
appsec (2.322 ms) : 2280, 2363
.   : milestone, 2322,
iast (2.073 ms) : 2021, 2125
.   : milestone, 2073,
iast_GLOBAL (2.126 ms) : 2073, 2179
.   : milestone, 2126,
profiling (1.944 ms) : 1902, 1986
.   : milestone, 1944,
tracing (1.916 ms) : 1876, 1957
.   : milestone, 1916,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.461 ms [1.45 ms, 1.472 ms]	-
appsec	2.321 ms [2.28 ms, 2.363 ms]	860.342 µs (58.9%)
iast	2.072 ms [2.019 ms, 2.124 ms]	610.801 µs (41.8%)
iast_GLOBAL	2.116 ms [2.063 ms, 2.169 ms]	655.073 µs (44.8%)
profiling	1.944 ms [1.902 ms, 1.986 ms]	483.273 µs (33.1%)
tracing	1.927 ms [1.887 ms, 1.968 ms]	466.096 µs (31.9%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.46 ms [1.448 ms, 1.471 ms]	-
appsec	2.322 ms [2.28 ms, 2.363 ms]	862.175 µs (59.1%)
iast	2.073 ms [2.021 ms, 2.125 ms]	613.325 µs (42.0%)
iast_GLOBAL	2.126 ms [2.073 ms, 2.179 ms]	666.677 µs (45.7%)
profiling	1.944 ms [1.902 ms, 1.986 ms]	484.487 µs (33.2%)
tracing	1.916 ms [1.876 ms, 1.957 ms]	456.878 µs (31.3%)

github-actions · 2024-12-20T16:29:00Z

Hi! 👋 Looks like you updated a Git Submodule.
If this was not intentional please make sure to:

Update the submodule to the latest commit on the master branch using the git submodule update command,
Check you properly set up your environment for contributing.

gary-huang force-pushed the gary/add-llmobs-configs branch from 3863387 to 19d2dc3 Compare December 10, 2024 21:32

gary-huang force-pushed the gary/add-llmobs-configs branch 2 times, most recently from fd136e0 to ba03bdd Compare December 20, 2024 16:22

add LLM obs configs

a8108c6

gary-huang force-pushed the gary/add-llmobs-configs branch from ba03bdd to a8108c6 Compare December 20, 2024 16:28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[MLOB] add LLM obs configs #8076

[MLOB] add LLM obs configs #8076

gary-huang commented Dec 10, 2024 •

edited

Loading

pr-commenter bot commented Dec 10, 2024 •

edited

Loading

github-actions bot commented Dec 20, 2024

[MLOB] add LLM obs configs #8076

Are you sure you want to change the base?

[MLOB] add LLM obs configs #8076

Conversation

gary-huang commented Dec 10, 2024 • edited Loading

What Does This Do

Motivation

Additional Notes

Contributor Checklist

pr-commenter bot commented Dec 10, 2024 • edited Loading

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

github-actions bot commented Dec 20, 2024

gary-huang commented Dec 10, 2024 •

edited

Loading

pr-commenter bot commented Dec 10, 2024 •

edited

Loading