Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion packages/docs/content/docs/02-sdk/07-intent-capture.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ The SDK reads and removes the `context` value at the protocol layer **before** y
- **Your handler never sees it.** Tool code and input schemas stay untouched — strict schemas included.
- **A call without `context` never fails.** The parameter is required only in the advertised schema; the server tolerates its absence.
- **Tools that define their own `context` parameter are left completely alone** — no injection, no capture, no stripping.
- **Widget-invoked tools get `context` as optional, automatically.** A widget iframe calls its tools with only their real arguments — it cannot know to send `context` — and hosts validate widget calls against the advertised schema, so a required `context` would make them refuse every such call. The failure is delayed (hosts cache connector schemas until a refresh), i.e. the widget would break silently some time *after* you enable intent capture. Three `_meta` shapes mark a tool as widget-invoked: `"openai/widgetAccessible": true`; an MCP Apps `ui.visibility` containing `"app"`; and — the broadest rule — a `ui.resourceUri` with **no** declared visibility, because MCP Apps defaults omitted visibility to `["model", "app"]`. That last rule covers every view-owning tool of every MCP App. If your widget never calls its view-owning tool, declare `ui.visibility: ["model"]` on it to keep `context` required there (maximum capture). The SDK logs one `[yavio] Intent context advertised as OPTIONAL…` line per affected tool so you can see exactly which tools were downgraded. Capture still works whenever a model call fills the optional field — but note that the `fallback` option is *not* invoked for context-less calls on these tools: they are presumptively the widget's own machine traffic, and inferring an intent per 3-second refresh would drown the real ones.

Captured intents are stored on the `tool_call` event (`intent_signals`), passed through the SDK's PII stripper, and capped at 500 characters. Read [what that does not cover](#what-intent-capture-does-not-protect-you-from) before enabling this on a server that handles sensitive data.

Expand Down Expand Up @@ -56,7 +57,7 @@ withYavio(server, {

## Dashboard status

The SDK reports its intent setting with the first tool call of each session, so the dashboard's **User Intents** panel always shows an honest state: *enabled*, *disabled*, or *SDK too old* (intent capture requires `@yavio/sdk` 0.2.0+). If capture is enabled but no intents arrive, the panel says so — some MCP clients do not fill unknown parameters.
The SDK reports its intent setting with the first tool call of each session, so the dashboard's **User Intents** panel always shows an honest state: *enabled*, *disabled*, or *SDK too old* (intent capture requires `@yavio/sdk` 0.2.0+). If capture is enabled but no intents arrive, check two causes before suspecting a bug: some MCP clients do not fill unknown parameters, and on **widget-invoked tools** `context` is optional by design (see above) — a tool that is mostly called by the app's own widget will naturally show few or no intents, because widget calls carry no conversation to describe. The SDK logs one `[yavio]` line per tool it downgrades, so the server log tells you which tools are affected.

## Privacy and app-store review

Expand Down
2 changes: 1 addition & 1 deletion packages/sdk/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@yavio/sdk",
"version": "0.3.1",
"version": "0.3.2",
"description": "Yavio SDK for instrumenting MCP servers with analytics, session tracking, and a React widget",
"type": "module",
"license": "MIT",
Expand Down
225 changes: 224 additions & 1 deletion packages/sdk/src/__tests__/integration/intent.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import { beforeEach, describe, expect, it, vi } from "vitest";
import { z } from "zod";
import { DEFAULT_INTENT_DESCRIPTION, resolveConfig } from "../../core/config.js";
import type { IntentConfig, YavioConfig } from "../../core/types.js";
import { MAX_INTENT_LENGTH } from "../../server/intent.js";
import { MAX_INTENT_LENGTH, createIntentController } from "../../server/intent.js";
import { _resetGlobalState, createProxy } from "../../server/proxy.js";
import type { Transport } from "../../transport/types.js";

Expand Down Expand Up @@ -782,3 +782,226 @@ describe("documentation stays in step with the shipped default", () => {
expect(page).toContain(DEFAULT_INTENT_DESCRIPTION);
});
});

describe("intent capture — widget-invoked tools", () => {
// A widget iframe calls its tools with only their real arguments; it cannot
// know to send `context`. Advertising `context` as REQUIRED on such a tool
// makes the host refuse every widget call as schema-invalid — and only after
// it refreshes cached schemas, so the widget breaks silently days later with
// no server-side trace. Incident: billiger-mietwagen, 2026-08-07 — the
// widget's 3s auto-refresh dropped from 83 calls/day to zero the moment the
// connector re-fetched schemas. Hence: on widget-invoked tools `context` is
// advertised as optional, while capture still applies when a model fills it.
beforeEach(() => _resetGlobalState());

const WIDGET_METAS: Array<[string, Record<string, unknown>]> = [

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WIDGET_METAS has seven positive cases and the suite has one negative (ui.visibility: ["model"]), but nothing pins { "openai/widgetAccessible": false }.

That leaves the strict === true check at intent.ts:337 unguarded — it's the obvious line for a later cleanup to "simplify" into a truthiness check. Behaviour would survive that change (false is still falsy), but the deliberate strictness would be lost with no failing test to explain why it was there. Given every other detection branch got a dedicated case, this is the one gap in the matrix.

["openai/widgetAccessible", { "openai/widgetAccessible": true }],
["nested ui.visibility array", { ui: { visibility: ["app"] } }],
["flat ui/visibility array", { "ui/visibility": ["app"] }],
["bare-string visibility (off-spec authoring slip)", { ui: { visibility: "app" } }],
[
"nested resourceUri with omitted visibility (spec default [model, app])",
{ ui: { resourceUri: "ui://views/x.html" } },
],
["flat ui/resourceUri with omitted visibility", { "ui/resourceUri": "ui://views/x.html" }],
[
"nested ui object beside a flat ui/visibility key (shadowing regression)",
{ ui: { resourceUri: "ui://views/x.html" }, "ui/visibility": ["app"] },
],
];

/** One registration shape for every widget test, plus a model-tool control. */
function setupWidget(
meta: Record<string, unknown>,
intent: IntentConfig = INTENT_ON,
onCall?: (args: unknown) => void,
): Promise<Harness> {
return setup(intent, (proxy) => {
proxy.registerTool(
"widget-refresh",
{ inputSchema: { search_id: z.string() }, _meta: meta },
async (args: { search_id: string }) => {
onCall?.(args);
return ok("refreshed");
},
);
proxy.registerTool("search", { inputSchema: { query: z.string() } }, async () => ok("x"));
});
}

it.each(WIDGET_METAS)("advertises context as optional: %s", async (_label, meta) => {
const h = await setupWidget(meta);
const list = await h.client.listTools();

const widgetTool = listedTool(list, "widget-refresh");
expect(widgetTool.inputSchema.properties).toHaveProperty("context");
expect(widgetTool.inputSchema.required ?? []).not.toContain("context");

// Control on the same server: ordinary tools keep the required parameter.
expect(listedTool(list, "search").inputSchema.required).toContain("context");
});

it("openai/widgetAccessible: false does not downgrade the tool", async () => {
// Pins the deliberate strict `=== true` check: an explicit false (or any
// non-true value) must not count as widget-invoked, and a later cleanup
// "simplifying" the comparison to truthiness has a failing test to answer.
const h = await setupWidget({ "openai/widgetAccessible": false });
const tool = listedTool(await h.client.listTools(), "widget-refresh");
expect(tool.inputSchema.required).toContain("context");
});

it("an explicit visibility of ['model'] keeps context required despite a resourceUri", async () => {
// The documented escape hatch: an app whose widget never calls its
// view-owning tool declares model-only visibility and keeps required
// context (maximum capture) on it.
const h = await setupWidget({
ui: { resourceUri: "ui://views/x.html", visibility: ["model"] },
});
const tool = listedTool(await h.client.listTools(), "widget-refresh");
expect(tool.inputSchema.required).toContain("context");
});

it("accepts a context-less widget call, records no intent, and skips the fallback", async () => {
// The fallback exists to approximate intents for model calls that omit
// context. Widget traffic (a 3s auto-refresh) is context-less by nature —
// running the fallback there would record machine-generated "inferred"
// intents at machine frequency.
const fallback = vi.fn(() => "machine noise");
let seenArgs: unknown;
const h = await setupWidget(
{ "openai/widgetAccessible": true },
{ ...INTENT_ON, fallback },
(args) => {
seenArgs = args;
},
);
await h.client.listTools();

const result = await h.client.callTool({
name: "widget-refresh",
arguments: { search_id: "abc123" },
});
expect(result.isError).toBeFalsy();
expect(seenArgs).toEqual({ search_id: "abc123" });
expect(toolCallEvents(h.events)[0]?.intent_signals).toBeUndefined();
expect(fallback).not.toHaveBeenCalled();

// The fallback still serves ordinary tools on the same server.
await h.client.callTool({ name: "search", arguments: { query: "boots" } });
expect(fallback).toHaveBeenCalledTimes(1);
expect(toolCallEvents(h.events)[1]?.intent_signals).toEqual({
intent: "machine noise",
source: "inferred",
});
});

it("still captures and strips context when a model call supplies it", async () => {
let seenArgs: unknown;
const h = await setupWidget({ "openai/widgetAccessible": true }, INTENT_ON, (args) => {
seenArgs = args;
});
await h.client.listTools();

const result = await h.client.callTool({
name: "widget-refresh",
arguments: { search_id: "abc123", context: "Fetching offer details the user asked about." },
});

expect(result.isError).toBeFalsy();
expect(seenArgs).toEqual({ search_id: "abc123" });
const event = toolCallEvents(h.events)[0];
expect(event?.intent_signals).toEqual({
intent: "Fetching offer details the user asked about.",
source: "context_parameter",
});
// The stripped context must not leak into input capture — the widget
// branch must assert no less than the model-tool test above.
expect(event?.input_values).not.toHaveProperty("context");
expect(event?.input_keys).not.toHaveProperty("context");
});
});

describe("intent capture — widget classification without listed _meta", () => {
// MCP SDK versions before 1.18.0 accept `_meta` in registerTool but drop
// it: it reaches neither the registry nor the tools/list entries. The
// registration-time record (noteToolRegistration's meta argument, fed by
// the proxy's registerTool interceptor) is then the only signal, so the
// wrapped list handler must classify from it even when the listed entry
// carries no _meta. Exercised against a stub low-level server because the
// MCP SDK installed in this repo always forwards _meta.
beforeEach(() => _resetGlobalState());

function installOnStub(intent: IntentConfig, listedTools: Array<Record<string, unknown>>) {
const handlers = new Map<string, (req: unknown, extra: unknown) => Promise<unknown>>();
handlers.set("tools/list", async () => ({ tools: listedTools }));
const controller = createIntentController(intent);
controller.install({
server: { setRequestHandler: () => {}, _requestHandlers: handlers },
_registeredTools: {},
} as unknown as McpServer);
return { controller, callList: () => handlers.get("tools/list")?.({}, {}) };
}

it("classifies from the registration-time _meta when the listed entry carries none", async () => {
const { controller, callList } = installOnStub(INTENT_ON, [
{
name: "widget-refresh",
inputSchema: { type: "object", properties: { search_id: { type: "string" } } },
},
{
name: "search",
inputSchema: { type: "object", properties: { query: { type: "string" } } },
},
]);
controller.noteToolRegistration("widget-refresh", [], { "openai/widgetAccessible": true });
controller.noteToolRegistration("search", []);

const result = (await callList()) as {
tools: Array<{ name: string; inputSchema: { properties: object; required?: string[] } }>;
};
const widget = result.tools.find((t) => t.name === "widget-refresh");
const search = result.tools.find((t) => t.name === "search");
expect(widget?.inputSchema.properties).toHaveProperty("context");
expect(widget?.inputSchema.required ?? []).not.toContain("context");
expect(search?.inputSchema.required).toContain("context");
});

it("removes a pre-existing 'context' entry from a widget tool's required array", async () => {
const { callList } = installOnStub(INTENT_ON, [
{
name: "widget-refresh",
_meta: { "openai/widgetAccessible": true },
inputSchema: {
type: "object",
properties: { search_id: { type: "string" } },
// Legal JSON Schema: required may name keys that properties omits —
// e.g. a stale leftover after the property itself was removed. Kept
// as-is it would defeat the widget exemption silently.
required: ["search_id", "context"],
},
},
]);
const result = (await callList()) as { tools: Array<{ inputSchema: { required?: string[] } }> };
expect(result.tools[0]?.inputSchema.required).toEqual(["search_id"]);
});

it("strips a stale required 'context' on ordinary tools under required: false", async () => {
// The stale-required cleanup is not widget-specific: with intent
// configured optional, a leftover required: ["context"] in a customer
// schema would otherwise ship `context` as required against the
// operator's explicit configuration — the hard-validating-client
// breakage `required: false` exists to avoid.
const { callList } = installOnStub({ ...INTENT_ON, required: false }, [
{
name: "search",
inputSchema: {
type: "object",
properties: { query: { type: "string" } },
required: ["query", "context"],
},
},
]);
const result = (await callList()) as { tools: Array<{ inputSchema: { required?: string[] } }> };
expect(result.tools[0]?.inputSchema.required).toEqual(["query"]);
});
});
2 changes: 1 addition & 1 deletion packages/sdk/src/core/version.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,4 @@
*
* Keep this in step with packages/sdk/package.json — a test asserts they match.
*/
export const SDK_VERSION = "0.3.1";
export const SDK_VERSION = "0.3.2";
6 changes: 5 additions & 1 deletion packages/sdk/src/server/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,12 @@ export function withYavio<T extends McpServer>(server: T, options?: WithYavioOpt
}

if (config.intent.enabled) {
// `required` is independently configurable; the log must not claim
// "required" for a server that was explicitly configured optional.
console.info(
"[yavio] Intent capture enabled: tools advertise a required 'context' parameter (pass intent: false to disable).",
config.intent.required
? "[yavio] Intent capture enabled: tools advertise a 'context' parameter — required on model-facing tools, optional on widget-invoked ones (pass intent: false to disable)."
: "[yavio] Intent capture enabled: tools advertise an optional 'context' parameter on every tool (pass intent: false to disable).",
);
}

Expand Down
Loading
Loading