Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions packages/agent/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@

- `toolFailureEnvelope` / `isToolFailureEnvelope` / `ToolFailureEnvelope` name the result details the loop attaches when a tool call fails without the tool returning details of its own. The guard matches only that envelope, so a consumer can tell it apart from a tool that reports a `failureKind` alongside its own details before dereferencing a tool-owned detail shape.

### Fixed

- The escaped-non-ASCII argument guard keeps its fail-closed terminal rejection and its unconditional two-resample budget for every tool and every field. After the budget is spent, a tool that enumerated its user-facing display fields (`displaySafeEscapedArgFields`; `ask` exempts only `questions.question` and `questions.options.label`) executes when every non-ASCII character lives inside those fields and is benign typographic punctuation (curated set: U+2014 em-dash). Escaped non-ASCII anywhere else — ids, deep-interview metadata, persisted records, non-ASCII object keys — and every other tool stays rejected terminally (#4627, reduced per both maintainer reviews: guard retained, exemption after budget and field-scoped).
## [0.14.0] - 2026-08-17

### Fixed
Expand Down
83 changes: 81 additions & 2 deletions packages/agent/src/agent-loop.ts
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,75 @@ function hasEscapedNonAsciiToolCall(message: AssistantMessage): boolean {
return message.content.some(block => block.type === "toolCall" && block.escapedNonAsciiArguments === true);
}

/**
* The complete set of non-ASCII characters an escaped payload may decode to and
* still execute on a display-safe tool after the resample budget: U+2014 EM DASH
* only. This is the exact character JSON encoders spell as `\u2014` that
* motivated the false positive — an English question text like "sessions —
* in-process?" with no other non-ASCII. Curated deliberately: a mistyped nibble
* here can only produce another punctuation/letter codepoint, and any such
* landing (or any other escaped character at all) keeps the fail-closed
* rejection. Currency, math, full-width, separator, letter, mark, number, and
* surrogate escapes all stay rejected everywhere.
*/
const DISPLAY_SAFE_ESCAPED_CODEPOINTS = new Set([0x2014]);

/** Structural type for tools that opt specific argument fields into display-safe handling. */
type DisplaySafeEscapedTool = AgentTool<TSchema> & {
/** Argument fields (dotted paths into the arguments object) that render to the user as display text. */
displaySafeEscapedArgFields?: readonly string[];
};

/**
* Whether a non-ASCII codepoint is benign typographic punctuation that a JSON
* encoder may escape in display text. See {@link DISPLAY_SAFE_ESCAPED_CODEPOINTS}.
*/
function isDisplaySafeEscapedCodepoint(cp: number): boolean {
return DISPLAY_SAFE_ESCAPED_CODEPOINTS.has(cp);
}

/**
* Walk the decoded arguments and decide whether the escaped payload is
* display-safe: every non-ASCII character must live inside one of the tool's
* declared display-field paths (dotted, array-index-free: `questions.question`
* matches every question in the `questions` array) AND be benign typographic
* punctuation. Any non-ASCII outside the display fields — ids, metadata,
* persisted records, or an object key — keeps the fail-closed rejection, as
* does any non-benign codepoint inside them.
*/
function isDisplaySafeEscapedArguments(tool: AgentTool<TSchema> | undefined, args: Record<string, unknown>): boolean {
const fields = (tool as DisplaySafeEscapedTool | undefined)?.displaySafeEscapedArgFields;
if (!fields || fields.length === 0) return false;
const prefixes = [...fields];
const isDisplayPath = (path: string): boolean =>
prefixes.some(field => path === field || path.startsWith(`${field}.`));
const walk = (node: unknown, path: string): boolean => {
if (typeof node === "string") {
for (const ch of node) {
const cp = ch.codePointAt(0);
if (cp === undefined || cp < 0x80) continue;
// Outside the display fields no non-ASCII is tolerated at all;
// inside them only the curated punctuation set is.
if (!isDisplayPath(path) || !isDisplaySafeEscapedCodepoint(cp)) return false;
}
return true;
}
if (Array.isArray(node)) return node.every(item => walk(item, path));
if (typeof node === "object" && node !== null) {
for (const [key, value] of Object.entries(node)) {
// Field names are structural identifiers; the display fields have
// fixed ASCII names, so a non-ASCII key is never exempted.
for (const ch of key) {
const cp = ch.codePointAt(0);
if (cp !== undefined && cp >= 0x80) return false;
}
if (!walk(value, path === "" ? key : `${path}.${key}`)) return false;
}
}
return true;
};
return walk(args, "");
}
/** Remove only the exact assistant response committed by its streaming attempt. */
function removeCommittedAssistantMessage(messages: AgentMessage[], message: AssistantMessage): boolean {
const index = messages.lastIndexOf(message);
Expand Down Expand Up @@ -2364,6 +2433,12 @@ async function runLoopBody(
// typed `escaped_arguments_discarded` outcome so the session policy
// owns a bounded same-model retry; the defect is never treated as
// provider evidence, so the fallback chain never advances on it.
//
// The budget runs unconditionally for escaped payloads: even a
// display-safe tool (whose terminal exemption lives in
// `executeToolCalls`) resamples here first, so the model always gets
// its chances to emit literal UTF-8 and the fallback stays rare
// rather than becoming the default path.
if (
message.stopReason !== "error" &&
message.stopReason !== "aborted" &&
Expand Down Expand Up @@ -3487,14 +3562,18 @@ async function executeToolCalls(
: `Tool call "${toolCall.name}" was cut off before its arguments finished streaming (the response hit its output token limit). The partial arguments cannot be executed. Re-issue the call with complete arguments, splitting the work into smaller steps if needed.`;
throw new Error(detail);
}
if (toolCall.escapedNonAsciiArguments) {
if (toolCall.escapedNonAsciiArguments && !isDisplaySafeEscapedArguments(tool, argsForExecution)) {
record.argumentValidationFailed = true;
// The arguments decoded cleanly, but they were spelled as `\uXXXX`
// escapes rather than literal UTF-8. Hand-written hex is where models
// mistype digits, and every mistyped nibble decodes to a different but
// equally valid character — the payload is unverifiable and cannot be
// repaired after parsing, so it is rejected rather than executed on
// silently corrupted text.
// silently corrupted text. The one bounded exception is a tool that
// declared its arguments display-only (see
// isDisplaySafeEscapedArguments): user-facing question text whose
// only non-ASCII characters are benign typographic punctuation is
// not the Hangul-mistype corruption this guard exists to stop.
throw new Error(
`Tool call "${toolCall.name}" spelled non-ASCII text as \\uXXXX escapes instead of literal UTF-8. ` +
`Escaped text cannot be verified — a single wrong hex digit silently becomes a different character — ` +
Expand Down
12 changes: 12 additions & 0 deletions packages/agent/src/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -698,6 +698,18 @@ export interface AgentTool<TParameters extends TSchema = TSchema, TDetails = any
* - function: `_i` is NOT injected; intent is derived dynamically from (potentially partial / streaming) args.
*/
intent?: "omit" | "optional" | "require" | ((args: Partial<Static<TParameters>>) => string | undefined);
/**
* Argument fields (dotted paths into the arguments object) that render to
* the user as pure display text — question wording and option labels, never
* ids, metadata, or persisted records. The agent loop uses this to bound the
* escaped-non-ASCII terminal rejection AFTER the resample budget is spent:
* when every non-ASCII character under these fields is benign typographic
* punctuation (e.g. an em-dash a JSON encoder escaped), the call executes
* instead of failing closed. Every other field of these arguments — and
* every field of every other tool — keeps the fail-closed rejection for
* unverified `\uXXXX` payloads.
*/
displaySafeEscapedArgFields?: readonly string[];

/** The main execution callback for this tool. */
execute: AgentToolExecFn<TParameters, TDetails, TTheme>;
Expand Down
216 changes: 216 additions & 0 deletions packages/agent/test/agent-loop-escaped-nonascii-toolcall.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,38 @@ function askTool(executed: Array<Record<string, unknown>>): AgentTool<typeof ask
};
}

/** A display-safe variant of the ask tool: opts its display field (question text) into the bounded exemption. */
function displaySafeAskTool(
executed: Array<Record<string, unknown>>,
): AgentTool<typeof askSchema, Record<string, never>> {
return {
...askTool(executed),
displaySafeEscapedArgFields: ["question"],
};
}

/** A stand-in for any mutating tool (write/edit/bash): never display-safe. */
function mutatingTool(executed: Array<Record<string, unknown>>): AgentTool<typeof askSchema, Record<string, never>> {
return {
...askTool(executed),
name: "write",
};
}

/** A display-safe escaped turn whose only non-ASCII is an em-dash. */
function emDashEscapedTurn(id: string, name = "ask") {
return {
content: [
{
type: "toolCall" as const,
id,
name,
arguments: { question: "How should the daemon drive sessions — in-process?" },
escapedNonAsciiArguments: true,
},
],
};
}
/** A turn whose raw arguments arrived spelled as `\uXXXX` instead of literal UTF-8. */
function escapedTurn(id: string, stopReason?: "aborted" | "error") {
return {
Expand Down Expand Up @@ -920,6 +952,190 @@ describe("agentLoop: ASCII-escaped non-ASCII argument guard", () => {
);
});

it("executes the benign em-dash ask case AFTER the resample budget on a display-safe tool", async () => {
const executed: Array<Record<string, unknown>> = [];
const context: AgentContext = { systemPrompt: [""], messages: [], tools: [displaySafeAskTool(executed)] };
// Three escaped wire attempts consume the budget (1 original + 2
// resamples); the third reaches terminal execution, where the
// field-scoped display-safe exemption lets the benign payload run.
const mock = createMockModel({
responses: [
emDashEscapedTurn("tc-1"),
emDashEscapedTurn("tc-2"),
emDashEscapedTurn("tc-3"),
{ content: ["done"] },
],
});
const config: AgentLoopConfig = { model: mock.model, convertToLlm: identityConverter };

const toolResults: Array<{ isError?: boolean; text: string }> = [];
const stream = agentLoop([createUserMessage("ask me")], context, config, undefined, mock.stream);
for await (const event of stream) {
if (event.type === "tool_execution_end") {
const first = event.result.content?.[0];
toolResults.push({ isError: event.isError, text: first?.type === "text" ? first.text : "" });
}
}

// The full budget ran before execution: 1 original + 2 resamples + the
// follow-up elicited by the executed tool result.
expect(mock.calls).toHaveLength(4);
expect(executed).toEqual([{ question: "How should the daemon drive sessions — in-process?" }]);
expect(toolResults).toHaveLength(1);
expect(toolResults[0].isError).toBeFalsy();
});

it("never exempts escaped non-ASCII outside the declared display fields", async () => {
const executed: Array<Record<string, unknown>> = [];
const context: AgentContext = { systemPrompt: [""], messages: [], tools: [displaySafeAskTool(executed)] };
// An em-dash in the QUESTION is benign; the same em-dash in a metadata
// field the tool did NOT enumerate (here: `deepInterview.dimension`)
// keeps the fail-closed rejection — the exemption is field-scoped.
const metaTurn = (id: string) => ({
content: [
{
type: "toolCall" as const,
id,
name: "ask",
arguments: {
question: "ok — fine",
deepInterview: { round: 1, component: "daemon", dimension: "스케줄 — 라우팅", ambiguity: 0.2 },
},
escapedNonAsciiArguments: true,
},
],
});
const mock = createMockModel({
responses: [metaTurn("tc-1"), metaTurn("tc-2"), metaTurn("tc-3"), { content: ["done"] }],
});
const config: AgentLoopConfig = { model: mock.model, convertToLlm: identityConverter };

const toolResults: Array<{ isError?: boolean; text: string }> = [];
const stream = agentLoop([createUserMessage("ask me")], context, config, undefined, mock.stream);
for await (const event of stream) {
if (event.type === "tool_execution_end") {
const first = event.result.content?.[0];
toolResults.push({ isError: event.isError, text: first?.type === "text" ? first.text : "" });
}
}

expect(executed).toHaveLength(0);
expect(toolResults).toHaveLength(1);
expect(toolResults[0].isError).toBe(true);
expect(toolResults[0].text).toContain("\\uXXXX");
});

it("never executes the same em-dash payload when the tool is not display-safe", async () => {
const executed: Array<Record<string, unknown>> = [];
const context: AgentContext = { systemPrompt: [""], messages: [], tools: [mutatingTool(executed)] };
const mock = createMockModel({
responses: [
emDashEscapedTurn("tc-1", "write"),
emDashEscapedTurn("tc-2", "write"),
emDashEscapedTurn("tc-3", "write"),
{ content: ["done"] },
],
});
const config: AgentLoopConfig = { model: mock.model, convertToLlm: identityConverter };

const toolResults: Array<{ isError?: boolean; text: string }> = [];
const stream = agentLoop([createUserMessage("ask me")], context, config, undefined, mock.stream);
for await (const event of stream) {
if (event.type === "tool_execution_end") {
const first = event.result.content?.[0];
toolResults.push({ isError: event.isError, text: first?.type === "text" ? first.text : "" });
}
}

// Mutating tools stay fail-closed: budget spent, then terminal rejection.
expect(executed).toHaveLength(0);
expect(toolResults).toHaveLength(1);
expect(toolResults[0].isError).toBe(true);
expect(toolResults[0].text).toContain("\\uXXXX");
});

it("rejects a nibble-adjacent symbol escape even on a display-safe tool", async () => {
const executed: Array<Record<string, unknown>> = [];
const context: AgentContext = { systemPrompt: [""], messages: [], tools: [displaySafeAskTool(executed)] };
const enDashTurn = (id: string) => ({
content: [
{
type: "toolCall" as const,
id,
name: "ask",
// U+2013 EN DASH — one nibble from the exempted U+2014.
arguments: { question: "range 0–1 inclusive?" },
escapedNonAsciiArguments: true,
},
],
});
const mock = createMockModel({
responses: [enDashTurn("tc-1"), enDashTurn("tc-2"), enDashTurn("tc-3"), { content: ["done"] }],
});
const config: AgentLoopConfig = { model: mock.model, convertToLlm: identityConverter };

const toolResults: Array<{ isError?: boolean; text: string }> = [];
const stream = agentLoop([createUserMessage("ask me")], context, config, undefined, mock.stream);
for await (const event of stream) {
if (event.type === "tool_execution_end") {
const first = event.result.content?.[0];
toolResults.push({ isError: event.isError, text: first?.type === "text" ? first.text : "" });
}
}

// The curated set admits only U+2014; every other symbol — including the
// nibble-adjacent en-dash — keeps the fail-closed rejection.
expect(executed).toHaveLength(0);
expect(toolResults).toHaveLength(1);
expect(toolResults[0].isError).toBe(true);
expect(toolResults[0].text).toContain("\\uXXXX");
});

it("rejects currency, math, full-width, separator, letter, and emoji escapes on a display-safe tool", async () => {
const redTeam: Array<[string, string]> = [
["currency ₩", "price ₩1,000?"],
["math ≈", "is x ≈ y?"],
["full-width !", "really!sure?"],
["ideographic space  ", "a b?"],
["hangul letter 안", "이름이 안 무엇인가?"],
["emoji 😀", "feeling 😀 today?"],
];
for (const [label, question] of redTeam) {
const executed: Array<Record<string, unknown>> = [];
const context: AgentContext = { systemPrompt: [""], messages: [], tools: [displaySafeAskTool(executed)] };
const turn = (id: string) => ({
content: [
{
type: "toolCall" as const,
id,
name: "ask",
arguments: { question },
escapedNonAsciiArguments: true,
},
],
});
const mock = createMockModel({
responses: [turn("tc-1"), turn("tc-2"), turn("tc-3"), { content: ["done"] }],
});
const config: AgentLoopConfig = { model: mock.model, convertToLlm: identityConverter };

const toolResults: Array<{ isError?: boolean; text: string }> = [];
const stream = agentLoop([createUserMessage("ask me")], context, config, undefined, mock.stream);
for await (const event of stream) {
if (event.type === "tool_execution_end") {
const first = event.result.content?.[0];
toolResults.push({ isError: event.isError, text: first?.type === "text" ? first.text : "" });
}
}

expect(executed).toHaveLength(0);
expect(toolResults).toHaveLength(1);
expect(toolResults[0].isError).toBe(true);
expect(toolResults[0].text).toContain("\\uXXXX");
expect(label).toBeTruthy();
}
});

it("executes literal UTF-8 arguments untouched", async () => {
const executed: Array<Record<string, unknown>> = [];
const context: AgentContext = { systemPrompt: [""], messages: [], tools: [askTool(executed)] };
Expand Down
1 change: 1 addition & 0 deletions packages/ai/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
- oMLX OpenAI-compatible completions now send `chat_template_kwargs.reasoning_effort` with `enable_thinking` when `thinkingFormat` is `qwen-chat-template`. Discovered oMLX models are treated as reasoning models with `low`/`medium`/`high` effort so local Qwen presets can differentiate roles without swapping weights.
- Fixed a resume-breaking HTTP 400 on `google-gemini-cli`/`google-antigravity` replay: assistant thinking blocks whose `thinkingSignature` is missing, empty (persistence clears oversized signatures to `""`), or invalid no longer emit an unsigned `{"thought": true}` part. Cloud Code Assist maps such parts to Anthropic `thinking` blocks and rejects the whole request with `messages.N.content.0.thinking.signature: Field required`, permanently bricking resumed sessions (#4630). Unsigned thinking now degrades to plain text — the same treatment cross-model reasoning already gets — while validly signed thinking still replays natively as a thought part with its `thoughtSignature`.
- `validateApiKeyAgainstModelsEndpoint` no longer accepts an API key on HTTP status alone. A 200 whose body is not JSON or carries no recognizable model list (OpenAI-compatible `data` array, gateway `models` array, or a bare array) now fails closed with an actionable error — previously a captive portal or broken gateway answering 200 with an HTML page silently validated and stored the key. Affects every provider using `kind: "models-endpoint"` (Synthetic, DeepSeek, DeepInfra, Fireworks, BizRouter, NanoGPT, OpenGateway, ZenMux, Fugu). Upstream bodies echoed into validation errors are now bounded to 200 characters on both validators.
- `ToolCall.escapedNonAsciiArguments` doc updated: the agent loop resamples unconditionally then rejects terminally, with a single bounded after-budget exception for tools that enumerated display fields (`displaySafeEscapedArgFields`) whose non-ASCII content is benign typographic punctuation. The scanner itself is unchanged and still flags every non-ASCII escape (#4627).

## [0.14.0] - 2026-08-17
- Cursor native tool calls (shell/read/write/… oneof variants) now convert their protobuf payloads into plain JSON-safe data before attaching them as toolCall `arguments`: `$typeName` markers are stripped, safe-range bigints become numbers (decimal strings beyond `Number.MAX_SAFE_INTEGER`), byte arrays become base64 strings, and cycles/functions collapse to null. Raw protobuf-es payloads carry `bigint` fields (`fileSize`, `durationMs`, `fileOutputThresholdBytes`, …) that defeat `JSON.stringify`, which broke managed snapshot staging, JSONL transcript persistence, and provider replay — the issue #4578 local-snapshot producer defect class fixed at its producer boundary.
Expand Down
Loading
Loading