Skip to content

Obtain JSR223 usage report after test execution #6510

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
KingRabbid wants to merge 7 commits into
base: master
Choose a base branch
from
Open

Obtain JSR223 usage report after test execution #6510

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
f719708
Allow more complex configuration of the cache; generate the usage rep…
KingRabbid Oct 6, 2025
1c82baa
Unify the JSR223 logging error statements
KingRabbid Oct 6, 2025
52282d8
Another set
KingRabbid Oct 6, 2025
c457940
Missing brackets
KingRabbid Oct 9, 2025
07af912
Merge branch 'refs/heads/jsr223-logging' into jsr223-cache-configurable
KingRabbid Oct 22, 2025
163b6f8
Merge branch 'refs/heads/master' into jsr223-cache-configurable
KingRabbid Oct 22, 2025
907fe10
Bump Caffeine to 3.1.8. Add Caffeine Javadoc link. Add debug logging …
KingRabbid Oct 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions bin/jmeter.properties
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1235,6 +1235,9 @@ view.results.tree.renderers_order=.RenderAsText,.RenderAsRegexp,.RenderAsBoundar
# Used by JSR-223 elements
# Size of compiled scripts cache
#jsr223.compiled_scripts_cache_size=100
# Configurable options on the compiled scripts cache, overrides the jsr223.compiled_scripts_cache_size property
# See com.github.benmanes.caffeine.cache.Caffeine API for details
jsr223.compiled_scripts_cache_spec=maximumSize=100,recordStats

#---------------------------------------------------------------------------
# Classpath configuration
Expand Down
16 changes: 8 additions & 8 deletions src/core/src/main/java/org/apache/jmeter/util/JSR223TestElement.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,10 +60,8 @@ public abstract class JSR223TestElement extends ScriptingTestElement
* Cache of compiled scripts
*/
private static final Cache<ScriptCacheKey, CompiledScript> COMPILED_SCRIPT_CACHE =
Caffeine
.newBuilder()
.maximumSize(JMeterUtils.getPropDefault("jsr223.compiled_scripts_cache_size", 100))
.build();
Caffeine.from(JMeterUtils.getPropDefault("jsr223.compiled_scripts_cache_spec","maximumSize=" +
JMeterUtils.getPropDefault("jsr223.compiled_scripts_cache_size", 100) + ",recordStats")).build();

/**
* Lambdas can't throw checked exceptions, so we wrap cache loading failure with a runtime one.
Expand Down Expand Up @@ -257,11 +255,11 @@ private static <T extends ScriptCacheKey> CompiledScript getCompiledScript(
} catch (ScriptCompilationInvocationTargetException e) {
Throwable cause = e.getCause();
if (cause instanceof IOException) {
cause.addSuppressed(new IllegalStateException("Unable to compile script " + newCacheKey));
cause.addSuppressed(new IllegalStateException("Unable to compile script: " + newCacheKey));
throw (IOException) cause;
}
if (cause instanceof ScriptException) {
cause.addSuppressed(new IllegalStateException("Unable to compile script " + newCacheKey));
cause.addSuppressed(new IllegalStateException("Unable to compile script: " + newCacheKey));
throw (ScriptException) cause;
}
throw e;
Expand All @@ -287,7 +285,7 @@ public boolean compile()
((Compilable) scriptEngine).compile(getScript());
return true;
} catch (ScriptException e) { // NOSONAR
logger.error("Error compiling script for test element {}, error:{}", getName(), e.getMessage());
logger.error("Error compiling script for test element named: '{}', error: {}", getName(), e.getMessage());
return false;
}
} else {
Expand All @@ -297,7 +295,7 @@ public boolean compile()
((Compilable) scriptEngine).compile(fileReader);
return true;
} catch (ScriptException e) { // NOSONAR
logger.error("Error compiling script for test element {}, error:{}", getName(), e.getMessage());
logger.error("Error compiling script for test element named: '{}', error: {}", getName(), e.getMessage());
return false;
}
}
Expand Down Expand Up @@ -357,6 +355,8 @@ public void testEnded() {
*/
@Override
public void testEnded(String host) {
if (COMPILED_SCRIPT_CACHE.estimatedSize() > 0)
logger.info("Compiled cache size: {}, stats: {}", COMPILED_SCRIPT_CACHE.estimatedSize(), COMPILED_SCRIPT_CACHE.stats());
Copy link
Collaborator

@vlsi vlsi Oct 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add braces. We do use them, however, the check was missing: #6516

Copy link
Collaborator

@vlsi vlsi Oct 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure the stats are actionable. The stats do not seem to help users to identify the offending elements.

What are the units for "Compiled cache size"?

Copy link
Contributor Author

@KingRabbid KingRabbid Oct 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi,

I've patched the code; the report only is the starting point of investigation, something which isn't available now? I'll come back in the following days with some sample JMX to help demo the outcome.
Indeed, doesn't point to exact "missing" JSR223 element, but still it's a starting point.
Disabling ${} is not necessary, maybe a warning is enough? Anyway, this matters only if the variable is stored in vars, if defined locally in the JSR223 then it works fine, i.e. is cached afaik.

Thanks

Copy link
Contributor Author

@KingRabbid KingRabbid Oct 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've pushed more changes; now I've implemented the logic to print, at debug level, JSR223 elements that don't get into cache: unchecked flag, compile problem.
Also, I've extended the output of the stats to include all relevant KPIs currently available in the API:
2025-10-23 00:40:00,693 INFO o.a.j.u.JSR223TestElement: JSR223 cached scripts: 2, requestsCount: 13 (hitCount: 4 + missedCount: 9), (hitRate: 0.31, missRate: 0.69), loadCount: 9 (loadSuccessCount: 2 + loadFailureCount: 7), evictionCount: 0, evictionWeight: 0, totalLoadTime: 1471.00 ms, averageLoadPenalty: 163.44 ms

Copy link
Contributor Author

@KingRabbid KingRabbid Oct 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Attaching sample JMX used for trials:
CCH-trials.zip
I've also updated the Caffeine library to v.3.1.8, latest 3.2.0 seems to bring incompatibilities or more changes because of dependencies.

COMPILED_SCRIPT_CACHE.invalidateAll();
scriptMd5 = null;
}
Expand Down
2 changes: 1 addition & 1 deletion xdocs/usermanual/component_reference.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1185,7 +1185,7 @@ To benefit from this feature:
</li>
</ul>
Cache size is controlled by the following JMeter property (<code>jmeter.properties</code>):
<source>jsr223.compiled_scripts_cache_size=100</source>
<source>jsr223.compiled_scripts_cache_size=100</source> or via a more complex cache setup using the <source>jsr223.compiled_scripts_cache_spec</source>
<note>Unlike the <complink name="BeanShell Sampler" />, the interpreter is not saved between invocations.</note>
<note>
JSR223 Test Elements using Script file or Script text + checked <code>Cache compiled script if available</code> are now compiled if ScriptEngine supports this feature, this enables great performance enhancements.
Expand Down
4 changes: 4 additions & 0 deletions xdocs/usermanual/properties_reference.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1985,6 +1985,10 @@ JMETER-SERVER</source>
Used by JSR-223 elements.<br/>
Size of compiled scripts cache.<br/>
Defaults to: <code>100</code></property>
<property name="jsr223.compiled_scripts_cache_spec">
Used by JSR-223 elements.<br/>
Caffeine framework spec configuration in String format. Overrides <code>jsr223.compiled_scripts_cache_size</code><br/>
Defaults to: <code>maximumSize=<jsr223.compiled_scripts_cache_size>,recordStats</code></property>
Copy link
Collaborator

@vlsi vlsi Oct 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you refer the documentation or copy the relevant bits here?

Copy link
Contributor Author

@KingRabbid KingRabbid Oct 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi,

some details in the Javadoc: https://www.javadoc.io/static/com.github.ben-manes.caffeine/caffeine/3.2.2/com.github.benmanes.caffeine/com/github/benmanes/caffeine/cache/Caffeine.html

Copy link
Contributor Author

@KingRabbid KingRabbid Oct 17, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spec/configurable attributes description: https://www.javadoc.io/doc/com.github.ben-manes.caffeine/caffeine/latest/com.github.benmanes.caffeine/com/github/benmanes/caffeine/cache/CaffeineSpec.html and logic behind counters: https://www.javadoc.io/doc/com.github.ben-manes.caffeine/caffeine/latest/com.github.benmanes.caffeine/com/github/benmanes/caffeine/cache/stats/CacheStats.html

Copy link
Collaborator

@vlsi vlsi Oct 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean it would be worth adding the links to JMeter documentation

Copy link
Contributor Author

@KingRabbid KingRabbid Oct 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've pushed new code, not sure it addresses your suggestion with regards to documentation, please let me know where to edit.

</properties>
</section>

Expand Down