Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion packages/docs/content/docs/02-sdk/07-intent-capture.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ The SDK reads and removes the `context` value at the protocol layer **before** y
- **Your handler never sees it.** Tool code and input schemas stay untouched — strict schemas included.
- **A call without `context` never fails.** The parameter is required only in the advertised schema; the server tolerates its absence.
- **Tools that define their own `context` parameter are left completely alone** — no injection, no capture, no stripping.
- **Widget-invoked tools get `context` as optional, automatically.** A tool marked as callable from an app widget (`_meta["openai/widgetAccessible"]: true`, or MCP Apps `ui.visibility` containing `"app"`) is invoked by the widget iframe with only its real arguments — an iframe cannot know to send `context`. Hosts validate widget calls against the advertised schema, so a required `context` would make them refuse every such call — and only once the host refreshes its cached schemas, i.e. the widget breaks silently some time *after* you enable intent capture. The SDK therefore never marks `context` as required on these tools. Capture still works whenever a model call fills the optional field.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This bullet documents two of the three detection rules but omits the broadest one and the only escape hatch.

Missing: a ui.resourceUri / ui/resourceUri with no visibility declared also downgrades the tool (intent.ts:346-347) — that's every view-owning tool of every MCP App, i.e. the widest-reaching rule here. Also missing: ui.visibility: ["model"], which both the code comment and the test call "the documented escape hatch" but which the docs never mention.

A customer whose intent coverage drops after upgrading currently has nothing here pointing at either the cause or the remedy — the same diagnostic dead end as the open Shipal coverage bug. Worth naming both, plus the [yavio] Intent context advertised as OPTIONAL… log line as the way to find affected tools.


Captured intents are stored on the `tool_call` event (`intent_signals`), passed through the SDK's PII stripper, and capped at 500 characters. Read [what that does not cover](#what-intent-capture-does-not-protect-you-from) before enabling this on a server that handles sensitive data.

Expand Down Expand Up @@ -56,7 +57,7 @@ withYavio(server, {

## Dashboard status

The SDK reports its intent setting with the first tool call of each session, so the dashboard's **User Intents** panel always shows an honest state: *enabled*, *disabled*, or *SDK too old* (intent capture requires `@yavio/sdk` 0.2.0+). If capture is enabled but no intents arrive, the panel says so — some MCP clients do not fill unknown parameters.
The SDK reports its intent setting with the first tool call of each session, so the dashboard's **User Intents** panel always shows an honest state: *enabled*, *disabled*, or *SDK too old* (intent capture requires `@yavio/sdk` 0.2.0+). If capture is enabled but no intents arrive, check two causes before suspecting a bug: some MCP clients do not fill unknown parameters, and on **widget-invoked tools** `context` is optional by design (see above) — a tool that is mostly called by the app's own widget will naturally show few or no intents, because widget calls carry no conversation to describe. The SDK logs one `[yavio]` line per tool it downgrades, so the server log tells you which tools are affected.

## Privacy and app-store review

Expand Down
2 changes: 1 addition & 1 deletion packages/sdk/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@yavio/sdk",
"version": "0.3.1",
"version": "0.3.2",
"description": "Yavio SDK for instrumenting MCP servers with analytics, session tracking, and a React widget",
"type": "module",
"license": "MIT",
Expand Down
196 changes: 195 additions & 1 deletion packages/sdk/src/__tests__/integration/intent.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import { beforeEach, describe, expect, it, vi } from "vitest";
import { z } from "zod";
import { DEFAULT_INTENT_DESCRIPTION, resolveConfig } from "../../core/config.js";
import type { IntentConfig, YavioConfig } from "../../core/types.js";
import { MAX_INTENT_LENGTH } from "../../server/intent.js";
import { MAX_INTENT_LENGTH, createIntentController } from "../../server/intent.js";
import { _resetGlobalState, createProxy } from "../../server/proxy.js";
import type { Transport } from "../../transport/types.js";

Expand Down Expand Up @@ -782,3 +782,197 @@ describe("documentation stays in step with the shipped default", () => {
expect(page).toContain(DEFAULT_INTENT_DESCRIPTION);
});
});

describe("intent capture — widget-invoked tools", () => {
// A widget iframe calls its tools with only their real arguments; it cannot
// know to send `context`. Advertising `context` as REQUIRED on such a tool
// makes the host refuse every widget call as schema-invalid — and only after
// it refreshes cached schemas, so the widget breaks silently days later with
// no server-side trace. Incident: billiger-mietwagen, 2026-08-07 — the
// widget's 3s auto-refresh dropped from 83 calls/day to zero the moment the
// connector re-fetched schemas. Hence: on widget-invoked tools `context` is
// advertised as optional, while capture still applies when a model fills it.
beforeEach(() => _resetGlobalState());

const WIDGET_METAS: Array<[string, Record<string, unknown>]> = [

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WIDGET_METAS has seven positive cases and the suite has one negative (ui.visibility: ["model"]), but nothing pins { "openai/widgetAccessible": false }.

That leaves the strict === true check at intent.ts:337 unguarded — it's the obvious line for a later cleanup to "simplify" into a truthiness check. Behaviour would survive that change (false is still falsy), but the deliberate strictness would be lost with no failing test to explain why it was there. Given every other detection branch got a dedicated case, this is the one gap in the matrix.

["openai/widgetAccessible", { "openai/widgetAccessible": true }],
["nested ui.visibility array", { ui: { visibility: ["app"] } }],
["flat ui/visibility array", { "ui/visibility": ["app"] }],
["bare-string visibility (off-spec authoring slip)", { ui: { visibility: "app" } }],
[
"nested resourceUri with omitted visibility (spec default [model, app])",
{ ui: { resourceUri: "ui://views/x.html" } },
],
["flat ui/resourceUri with omitted visibility", { "ui/resourceUri": "ui://views/x.html" }],
[
"nested ui object beside a flat ui/visibility key (shadowing regression)",
{ ui: { resourceUri: "ui://views/x.html" }, "ui/visibility": ["app"] },
],
];

/** One registration shape for every widget test, plus a model-tool control. */
function setupWidget(
meta: Record<string, unknown>,
intent: IntentConfig = INTENT_ON,
onCall?: (args: unknown) => void,
): Promise<Harness> {
return setup(intent, (proxy) => {
proxy.registerTool(
"widget-refresh",
{ inputSchema: { search_id: z.string() }, _meta: meta },
async (args: { search_id: string }) => {
onCall?.(args);
return ok("refreshed");
},
);
proxy.registerTool("search", { inputSchema: { query: z.string() } }, async () => ok("x"));
});
}

it.each(WIDGET_METAS)("advertises context as optional: %s", async (_label, meta) => {
const h = await setupWidget(meta);
const list = await h.client.listTools();

const widgetTool = listedTool(list, "widget-refresh");
expect(widgetTool.inputSchema.properties).toHaveProperty("context");
expect(widgetTool.inputSchema.required ?? []).not.toContain("context");

// Control on the same server: ordinary tools keep the required parameter.
expect(listedTool(list, "search").inputSchema.required).toContain("context");
});

it("an explicit visibility of ['model'] keeps context required despite a resourceUri", async () => {
// The documented escape hatch: an app whose widget never calls its
// view-owning tool declares model-only visibility and keeps required
// context (maximum capture) on it.
const h = await setupWidget({
ui: { resourceUri: "ui://views/x.html", visibility: ["model"] },
});
const tool = listedTool(await h.client.listTools(), "widget-refresh");
expect(tool.inputSchema.required).toContain("context");
});

it("accepts a context-less widget call, records no intent, and skips the fallback", async () => {
// The fallback exists to approximate intents for model calls that omit
// context. Widget traffic (a 3s auto-refresh) is context-less by nature —
// running the fallback there would record machine-generated "inferred"
// intents at machine frequency.
const fallback = vi.fn(() => "machine noise");
let seenArgs: unknown;
const h = await setupWidget(
{ "openai/widgetAccessible": true },
{ ...INTENT_ON, fallback },
(args) => {
seenArgs = args;
},
);
await h.client.listTools();

const result = await h.client.callTool({
name: "widget-refresh",
arguments: { search_id: "abc123" },
});
expect(result.isError).toBeFalsy();
expect(seenArgs).toEqual({ search_id: "abc123" });
expect(toolCallEvents(h.events)[0]?.intent_signals).toBeUndefined();
expect(fallback).not.toHaveBeenCalled();

// The fallback still serves ordinary tools on the same server.
await h.client.callTool({ name: "search", arguments: { query: "boots" } });
expect(fallback).toHaveBeenCalledTimes(1);
expect(toolCallEvents(h.events)[1]?.intent_signals).toEqual({
intent: "machine noise",
source: "inferred",
});
});

it("still captures and strips context when a model call supplies it", async () => {
let seenArgs: unknown;
const h = await setupWidget({ "openai/widgetAccessible": true }, INTENT_ON, (args) => {
seenArgs = args;
});
await h.client.listTools();

const result = await h.client.callTool({
name: "widget-refresh",
arguments: { search_id: "abc123", context: "Fetching offer details the user asked about." },
});

expect(result.isError).toBeFalsy();
expect(seenArgs).toEqual({ search_id: "abc123" });
const event = toolCallEvents(h.events)[0];
expect(event?.intent_signals).toEqual({
intent: "Fetching offer details the user asked about.",
source: "context_parameter",
});
// The stripped context must not leak into input capture — the widget
// branch must assert no less than the model-tool test above.
expect(event?.input_values).not.toHaveProperty("context");
expect(event?.input_keys).not.toHaveProperty("context");
});
});

describe("intent capture — widget classification without listed _meta", () => {
// MCP SDK versions before 1.18.0 accept `_meta` in registerTool but drop
// it: it reaches neither the registry nor the tools/list entries. The
// registration-time record (noteToolRegistration's meta argument, fed by
// the proxy's registerTool interceptor) is then the only signal, so the
// wrapped list handler must classify from it even when the listed entry
// carries no _meta. Exercised against a stub low-level server because the
// MCP SDK installed in this repo always forwards _meta.
beforeEach(() => _resetGlobalState());

function installOnStub(intent: IntentConfig, listedTools: Array<Record<string, unknown>>) {
const handlers = new Map<string, (req: unknown, extra: unknown) => Promise<unknown>>();
handlers.set("tools/list", async () => ({ tools: listedTools }));
const controller = createIntentController(intent);
controller.install({
server: { setRequestHandler: () => {}, _requestHandlers: handlers },
_registeredTools: {},
} as unknown as McpServer);
return { controller, callList: () => handlers.get("tools/list")?.({}, {}) };
}

it("classifies from the registration-time _meta when the listed entry carries none", async () => {
const { controller, callList } = installOnStub(INTENT_ON, [
{
name: "widget-refresh",
inputSchema: { type: "object", properties: { search_id: { type: "string" } } },
},
{
name: "search",
inputSchema: { type: "object", properties: { query: { type: "string" } } },
},
]);
controller.noteToolRegistration("widget-refresh", [], { "openai/widgetAccessible": true });
controller.noteToolRegistration("search", []);

const result = (await callList()) as {
tools: Array<{ name: string; inputSchema: { properties: object; required?: string[] } }>;
};
const widget = result.tools.find((t) => t.name === "widget-refresh");
const search = result.tools.find((t) => t.name === "search");
expect(widget?.inputSchema.properties).toHaveProperty("context");
expect(widget?.inputSchema.required ?? []).not.toContain("context");
expect(search?.inputSchema.required).toContain("context");
});

it("removes a pre-existing 'context' entry from a widget tool's required array", async () => {
const { callList } = installOnStub(INTENT_ON, [
{
name: "widget-refresh",
_meta: { "openai/widgetAccessible": true },
inputSchema: {
type: "object",
properties: { search_id: { type: "string" } },
// Legal JSON Schema: required may name keys that properties omits —
// e.g. a stale leftover after the property itself was removed. Kept
// as-is it would defeat the widget exemption silently.
required: ["search_id", "context"],
},
},
]);
const result = (await callList()) as { tools: Array<{ inputSchema: { required?: string[] } }> };
expect(result.tools[0]?.inputSchema.required).toEqual(["search_id"]);
});
});
2 changes: 1 addition & 1 deletion packages/sdk/src/core/version.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,4 @@
*
* Keep this in step with packages/sdk/package.json — a test asserts they match.
*/
export const SDK_VERSION = "0.3.1";
export const SDK_VERSION = "0.3.2";
2 changes: 1 addition & 1 deletion packages/sdk/src/server/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ export function withYavio<T extends McpServer>(server: T, options?: WithYavioOpt

if (config.intent.enabled) {
console.info(
"[yavio] Intent capture enabled: tools advertise a required 'context' parameter (pass intent: false to disable).",
"[yavio] Intent capture enabled: tools advertise a 'context' parameter — required on model-facing tools, optional on widget-invoked ones (pass intent: false to disable).",

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This log is gated only on config.intent.enabled, but required is independently configurable (intent: { required: false }, documented at 07-intent-capture.mdx). A server started that way advertises context as optional on every tool while this line claims it is "required on model-facing tools".

The previous wording was vaguer and therefore accidentally correct; making it specific made it wrong in that configuration. Suggest branching on config.intent.required, or dropping back to naming only the widget exemption.

);
}

Expand Down
Loading
Loading