Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/trigger-deploy
Original file line number Diff line number Diff line change
@@ -1 +1 @@
1786027990
1786099674
2 changes: 1 addition & 1 deletion .github/trigger-test
Original file line number Diff line number Diff line change
@@ -1 +1 @@
1786029809
1786099674
6 changes: 6 additions & 0 deletions .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1388,6 +1388,9 @@ jobs:
- name: A registry listing is not a seller answering (origin-responded)
run: node scripts/test-origin-responded.js

- name: "A venue with many payees is not one seller (multi-payTo index — offline)"
run: node scripts/test-multi-payto.js

- name: A settlement rejection is not a charged failure (public stats honesty)
run: STATS_ALLOW_EPHEMERAL=true node scripts/test-charged-failure-honesty.js

Expand Down Expand Up @@ -1448,6 +1451,9 @@ jobs:
- name: MCP search_tools — live-catalog smoke test (locks top-1 for agent queries)
run: node scripts/test-mcp-search-ranking.js

- name: "Self-consistency — every tool and route our own text names must exist"
run: TARGET_URL=http://localhost:3000 node scripts/test-mcp-self-consistency.js

- name: Exercise tools against live sites
run: |
echo "=== extract: BBC News article ==="
Expand Down
32 changes: 32 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -300,6 +300,38 @@ with `res.statusCode === 200`. (`node_modules/@x402/express/dist/esm/index.mjs`.
server.js now rewrites HEAD on catalog GET routes to GET for the gate chain
and suppresses the body at res.end (RFC 9110 semantics: 402 + identical
headers, empty body). `scripts/test-head-paywall.js` (offline, in CI).
- **Surface self-consistency (`scripts/test-mcp-self-consistency.js`, 2026-08-07, in CI):**
every functional test drives the connector the way WE intend it to be used — they
call the tools whose names they already know — so nothing tested whether our own
published text names things that EXIST. Three times a tool had a working CallTool
handler and was absent from `tools/list` (about_agent402, top_x402_sellers, then
`request_tool`); the first two were fixed by hand with a comment and no test, so the
class stayed open and the third shipped. The third was the worst: about_agent402's
`missingATool` field tells agents to "Call request_tool", i.e. our orientation tool
instructed agents to do something our capabilities made impossible, and the whole
demand board only ever heard from callers who already knew the name. Found from
OUTSIDE (issue #705), not by us. The guard reads five agent-facing surfaces
(`tools/list` text, about_agent402, get_payment_info, `/llms.txt`,
`/.well-known/x402`) and asserts every tool name in a call-this position is
advertised, every named catalog slug exists, and every referenced route is
registered — plus both parity directions (a CallTool branch no advertised name can
reach; a listed tool with no handler or slug). **Route existence uses TWO oracles
and reports missing only when BOTH say no:** a source scan of `app.<verb>("…")`
(the only oracle that can see a POST-only route — a live GET 404 cannot
distinguish "no such route" from "wrong method", the ambiguity the #705 reporter
correctly refused to resolve) and a live GET (the only oracle that can see the
template-literal chain pages `app.get(\`/${chainKey}\`)`). The live probe never
touches `/api` or `/v1` — in FREE_MODE those handlers execute, and a consistency
check must not call a tool that spends money. Path matching is SEGMENT-aware so
`/api/wish` is never satisfied by `/api/wishes`. Extractors are proven against a
planted control before any clean run is believed (same doctrine as the free-tier
egress probe). **The first draft had a "does this look like one of our tools?"
filter that skipped any unknown snake_case name — which is exactly the defect
being hunted; a planted `Call submit_wish` passed a green run.** It is gone; the
only escape hatch is the explicit `NOT_A_TOOL` set (one entry: `route_and_execute`,
which is real but lives on the stdio npm package). Mutation-tested: removing
`request_tool` from the listing fails 2 assertions, a fake tool name fails 1, a
fake route fails 1.
- **Marketplace latency / snapshot caching (`src/x402-economy.js`):** `GET /marketplace`
(and `/api/x402-economy`) render from `x402EconomySnapshot()` — a ~500ms on-chain read
(EIP-3009 USDC settlements on Base via CDP SQL). It is **stale-while-revalidate**: a fresh
Expand Down
30 changes: 23 additions & 7 deletions scripts/test-mcp-http.js
Original file line number Diff line number Diff line change
Expand Up @@ -42,26 +42,42 @@ const names = (list.result?.tools ?? []).map((t) => t.name).sort();
// uniformly verb_noun (action-first) — the pattern Glama's naming-consistency
// rubric names as the target. Every prior spelling still routes.
//
// Now 17, not 15: about_agent402 and top_x402_sellers had working handlers but
// were absent from tools/list, which is the ONLY surface an MCP client can
// discover from — and the service manifest already advertised
// Now 18, not 15: about_agent402, top_x402_sellers and request_tool had working
// handlers but were absent from tools/list, which is the ONLY surface an MCP
// client can discover from — and the service manifest already advertised
// top_x402_sellers, so it was promised and unreachable. Listing them trades
// two tools of directory-scoring headroom for capabilities that were being
// three tools of directory-scoring headroom for capabilities that were being
// paid for in documentation and not delivered. Keep this list tight: anything
// further belongs behind call_tool.
//
// request_tool (added for issue #705, reported from OUTSIDE) was the third
// instance of that one defect, and the worst: about_agent402's own text tells
// agents to call it. Fixing instances three times without pinning the class is
// why a stranger found it before we did — scripts/test-mcp-self-consistency.js
// now fails on any handler that no advertised name can reach.
const EXPECTED_LIST = [
"search_tools", "find_tool", "call_tool", "get_payment_info",
"generate_hash", "convert_units", "generate_qr", "format_json", "decode_jwt", "convert_base64", "generate_uuid", "parse_csv", "convert_timezone",
"get_wallet_balances", "get_wallet_transactions",
"about_agent402", "top_x402_sellers",
"about_agent402", "top_x402_sellers", "request_tool",
].sort();
assert(
names.length === EXPECTED_LIST.length && EXPECTED_LIST.every((n) => names.includes(n)),
`tools/list is the curated set (got ${names.length}: ${names.join(",")})`
);
assert(
(list.result?.tools ?? []).every((t) => t.title && t.annotations?.readOnlyHint === true),
"every tool carries a title + read-only safety annotations (directory requirement)"
(list.result?.tools ?? []).every((t) => t.title && typeof t.annotations?.readOnlyHint === "boolean"),
"every tool carries a title + safety annotations (directory requirement)"
);
// The annotations must be TRUE, not merely present. request_tool records a wish
// row, so it is the one tool here that is not read-only; a client that trusts
// readOnlyHint to decide what it may call unattended would be misled by a
// blanket true. Equally, a read-only tool silently flipping to false would cost
// us calls from exactly those clients.
const writers = (list.result?.tools ?? []).filter((t) => t.annotations?.readOnlyHint === false).map((t) => t.name).sort();
assert(
writers.length === 1 && writers[0] === "request_tool",
`request_tool is the only tool annotated as a writer (got ${writers.join(",") || "none"})`
);

// Renames must never break an existing caller: the current verb_noun name, the
Expand Down
Loading