Add escape output parameter #4179

Hailong-am · 2025-09-18T02:47:07Z

Description

Add a new parameter to escape tool output,so that the tool output is escaped and able to used in MLModelTool as part of prompt

Related Issues

#4178

Check List

New functionality includes testing.
New functionality has been documented.
API changes companion pull request created.
Commits are signed per the DCO using --signoff.
Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Hailong Cui <[email protected]>

xinyual · 2025-09-18T03:24:47Z

Is this mainly for flow agent?

Hailong-am · 2025-09-18T03:28:50Z

Is this mainly for flow agent?

Yes, that's right.

codecov · 2025-09-23T08:47:02Z

Codecov Report

❌ Patch coverage is 70.00000% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.03%. Comparing base (247a88c) to head (97c269a).

Files with missing lines	Patch %	Lines
...orithms/agent/MLConversationalFlowAgentRunner.java	0.00%	3 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #4179      +/-   ##
============================================
+ Coverage     82.01%   82.03%   +0.01%     
- Complexity     9183     9197      +14     
============================================
  Files           789      789              
  Lines         39567    39574       +7     
  Branches       4389     4390       +1     
============================================
+ Hits          32450    32463      +13     
+ Misses         5227     5221       -6     
  Partials       1890     1890

Flag	Coverage Δ
ml-commons	`82.03% <70.00%> (+0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

common/src/main/java/org/opensearch/ml/common/utils/ToolUtils.java

Signed-off-by: Hailong Cui <[email protected]>

Hailong-am · 2025-09-24T03:32:08Z

Run the unit test and integTest locally. Both are pass

➜  ml-commons git:(chatbot_invalidpaylod) ✗ ./gradlew build
=======================================
OpenSearch Build Hamster says Hello!
  Gradle Version        : 8.14.3
  OS Info               : Linux 5.10.239-216.958.amzn2int.x86_64 (amd64)
  JDK Version           : 21 (Amazon Corretto JDK)
  JAVA_HOME             : /usr/lib/jvm/java-21-amazon-corretto
  Random Testing Seed   : DE666DBB1CBF3C3B
  Crypto Standard       : any-supported
=======================================

....

BUILD SUCCESSFUL in 27m 10s
103 actionable tasks: 55 executed, 48 up-to-date

akolarkunnu · 2025-09-24T10:28:45Z

common/src/main/java/org/opensearch/ml/common/utils/StringUtils.java

     * @return
     */
-    public static String prepareJsonValue(String input) {
+    public static String prepareJsonValue(String input, boolean escape) {


You need to change API documentation based on new behavior

right, the documentation need update as well

akolarkunnu · 2025-09-24T12:13:38Z

common/src/test/java/org/opensearch/ml/common/utils/StringUtilsTest.java

    public void prepareJsonValue_escapesBadCharsOtherwise() {
        String input = "Tom & Jerry \"<script>";
-        String escaped = StringUtils.prepareJsonValue(input);
+        String escaped = StringUtils.prepareJsonValue(input, false);


Something is misleading here, you are actually escaping the plain text, but passing "escape" parameter as false. May be rename "escape" parameter in the API definition, which should understand it is only about json string.

make sense, i will update it

rename it to escapeJson

dhrubo-os · 2025-09-24T19:55:17Z

...src/main/java/org/opensearch/ml/engine/algorithms/agent/MLConversationalFlowAgentRunner.java

+        params
+            .put(
+                outputKey,
+                StringUtils.prepareJsonValue(filteredOutput, Boolean.parseBoolean(toolParameters.getOrDefault(TOOL_ESCAPE_OUTPUT, "false")))


are we expecting customer to provide this settings: TOOL_ESCAPE_OUTPUT?

yes, as tool parameters whebln configure the agent.

An example

"user_inputs": { "description": "Agent Tool", "include_output_in_agent_response": True, "type": "AgentTool", "parameters": { "max_iteration": "10", "escape_output": "true" }, "name": "AgentTool" }

Just trying to understand, based on what condition/experience customer will try to set True/False on this parameter? How will customer determine if they need to set this or not? This seems too much details for customer to understand.

that depends on the use case, if the tool output will used to concat with other strings to build a prompt for LLM, that's a typical case we need to set it as true. That is the case for chatbot question suggestor which is a MLModelTool and leverage previous tool output to build the prompt for LLM

Just trying to understand, based on what condition/experience customer will try to set True/False on this parameter? How will customer determine if they need to set this or not? This seems too much details for customer to understand.

+1

Signed-off-by: Hailong Cui <[email protected]>

Hailong-am · 2025-09-25T09:46:14Z

CI failed due to model throttling

RestMLRAGSearchProcessorIT > testBM25WithBedrock FAILED
    org.opensearch.client.ResponseException: method [POST], host [http://127.0.0.1:60288/], URI [/test/_search?size=5&search_pipeline=pipeline_test], status line [HTTP/1.1 429 Too Many Requests]
    {"error":{"root_cause":[{"type":"remote_connector_throttling_exception","reason":"Error from remote service: The request was denied due to remote server throttling. To change the retry policy and behavior, please update the connector client_config."}],"type":"remote_connector_throttling_exception","reason":"Error from remote service: The request was denied due to remote server throttling. To change the retry policy and behavior, please update the connector client_config."},"status":429}

pyek-bot · 2025-09-30T01:34:45Z

Hi hailong, do you think this PR will help address the current issue: #4093

pyek-bot

Functionality looks good! Please address the other comments and add coverage if required.

add escape output parameter

14bcb07

Signed-off-by: Hailong Cui <[email protected]>

Hailong-am force-pushed the chatbot_invalidpaylod branch from c55e49a to 14bcb07 Compare September 18, 2025 02:51

Hailong-am requested a deployment to ml-commons-cicd-env-require-approval September 18, 2025 02:52 — with GitHub Actions Waiting

Hailong-am marked this pull request as ready for review September 18, 2025 03:13

Hailong-am requested review from b4sjoo, dhrubo-os, mingshl, jngz-es, model-collapse, rbhavna, ylwu-amzn, zane-neo, Zhangxunmt, austintlee, HenryL27, sam-herman, xinyual and pyek-bot as code owners September 18, 2025 03:13

xinyual previously approved these changes Sep 18, 2025

View reviewed changes

Merge branch 'main' into chatbot_invalidpaylod

c824105

Hailong-am temporarily deployed to ml-commons-cicd-env-require-approval September 23, 2025 07:28 — with GitHub Actions Inactive

Hailong-am requested a deployment to ml-commons-cicd-env-require-approval September 23, 2025 08:48 — with GitHub Actions Waiting

pyek-bot reviewed Sep 23, 2025

View reviewed changes

common/src/main/java/org/opensearch/ml/common/utils/ToolUtils.java Outdated Show resolved Hide resolved

Hailong-am dismissed xinyual’s stale review via 870a9a9 September 24, 2025 02:12

Hailong-am requested a deployment to ml-commons-cicd-env-require-approval September 24, 2025 02:13 — with GitHub Actions Waiting

rename parameter name

97c269a

Signed-off-by: Hailong Cui <[email protected]>

Hailong-am force-pushed the chatbot_invalidpaylod branch from 870a9a9 to 97c269a Compare September 24, 2025 02:13

Hailong-am temporarily deployed to ml-commons-cicd-env-require-approval September 24, 2025 02:15 — with GitHub Actions Inactive

Hailong-am requested a deployment to ml-commons-cicd-env-require-approval September 24, 2025 03:35 — with GitHub Actions Waiting

akolarkunnu reviewed Sep 24, 2025

View reviewed changes

dhrubo-os reviewed Sep 24, 2025

View reviewed changes

update parameter name

1192c34

Signed-off-by: Hailong Cui <[email protected]>

Hailong-am had a problem deploying to ml-commons-cicd-env-require-approval September 25, 2025 00:33 — with GitHub Actions Error

Hailong-am had a problem deploying to ml-commons-cicd-env-require-approval September 25, 2025 00:33 — with GitHub Actions Failure

Hailong-am had a problem deploying to ml-commons-cicd-env-require-approval September 25, 2025 00:33 — with GitHub Actions Error

pyek-bot approved these changes Sep 30, 2025

View reviewed changes

Add escape output parameter #4179

Are you sure you want to change the base?

Add escape output parameter #4179

Uh oh!

Conversation

Hailong-am commented Sep 18, 2025

Description

Related Issues

Check List

Uh oh!

xinyual commented Sep 18, 2025

Uh oh!

Hailong-am commented Sep 18, 2025

Uh oh!

codecov bot commented Sep 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Hailong-am commented Sep 24, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Hailong-am Sep 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Hailong-am commented Sep 25, 2025

Uh oh!

pyek-bot commented Sep 30, 2025

Uh oh!

pyek-bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

codecov bot commented Sep 23, 2025 •

edited

Loading

Hailong-am Sep 25, 2025 •

edited

Loading