mcp: surface unknown tool options at dispatch - #614
Conversation
Dispatch read only the keys it knew, so an unknown option vanished silently — read_file(line_range: ...) returned the full file at maximum token cost with no signal to self-correct (zzet#597). Close every structured schema that never took a position, publish that in tools/list, and enforce it before the handler runs: unknown keys error immediately, naming them and the valid options. GORTEX_TOOL_ARG_GUARD=warn appends an _ignored_options rider instead; =off restores the old behavior. Facade names stay exempt — their compat wrapper accepts legacy shapes by design.
zzet
left a comment
There was a problem hiding this comment.
Requesting changes. The blanket additionalProperties:false stamp closes 152 of 173 registered tools and hard-rejects keys gortex's own callers inject. Verified on 277152b against b55b9a0.
1. format injection → refusal on 54/152 closed tools
Three first-party injectors add format to arbitrary tools' args:
cmd/gortex/cli_daemon.go:95—buildToolCallFrameWithDefaultpinsformat:"json"into every legacy-surface CLI frame (pinJSONDefault = tools != facade-v1; CLI verbs requestcore).cmd/gortex/call.go:117—gortex call <non-facade tool>setsargObj["format"] = callFormat(defaultjson). Only the facade branch is exempt.internal/server/handler.go:449— merges?format=/ bodyformatinto any tool.docs/server.md:32documents this for "any MCP tool".
Broken:
gortex audit→audit_healthgortex affected→get_test_targets(affected.go:149)gortex tools list|receipt→tool_profile(declares only["tool"]). The receipt reportsAdvertisedTools:0 / no_surface_mountedagainst a healthy daemon, no error surfaced.gortex edit— 9/13 subcommands viaedit.go:67:verify_change,get_edit_plan,preview_edit,simulate_chain,batch_edit,edit_file,edit_symbol,rename_symbol,get_test_targetsgortex call edit_file --arg path=...— the invocation printed indocs/cli.md:250POST /v1/tools/read_file?format=gcxand 53 other tools
audit_health does not accept option(s): format; valid options: project, repo, scope, workspace
2. Handler-honored keys that no schema declares
read_filereadsmax_chars(tools_fileops.go:1408, honored viacapReadFileContent) but doesn't declare it.TestReadFilePhysicalEvidenceMaxCharsRetainsTruncationContractasserts that contract and stays green. Its description also saysComposable with format:"gcx"while declaring noformat.budget.go:481effectiveBudgetreadsmax_bytes/max_tokensgenerically for every tool (budget.go:55: "wired onto every list-shaped tool");applyFieldsFilterdoes the same forfields. 104/152 closed tools don't declaremax_bytes, 148/152 don't declarefields.search_text{max_bytes}andgraph_query{fields}now hard-error.
3. Gortex's own hook self-degrades silently
internal/hooks/subagent.go renderTaskContext calls smart_context{task, compact:true}. smart_context declares no compact. Both hook transports collapse isError to "", so the ### Relevant Symbols block drops out of every subagent briefing with nothing logged.
4. The suite cannot see any of this
Full suite is green on the head (internal/mcp, internal/server, internal/server/hub, cmd/gortex). Nothing crosses the guarded path with a caller-built arg map:
internal/mcp/server_test.go:69findAndCallHandlerdispatches through a hand-maintained handler map, bypassingprepareTool.cmd/gortextests stub the relay seam (edit_test.go:40,tools_cmd_test.go:38,analyze_test.go:31).cli_daemon_test.go:13only asserts the pin exists, usingsearch_symbols/graph_stats— both declareformat.internal/server/handler_test.go:204's?format=gcxtest registers its fixture via baremcpserver.AddTool.
5. Guard is inert where #597 was observed
docs/mcp.md:114: every connection with a non-empty clientInfo.name defaults to facade-v1 — the 21 tools exempted at server.go:3120. Second cause: s.facades.capture(*tool, handler) at :3128 stores the pre-guard closure facade dispatch invokes. Measured: read(operation:"file", target:{file:"main.go"}, line_range:"1-3") returns the whole file, isError=false, no rider. So the fix doesn't fire where the bug was reported, and does fire on the CLI, which it breaks.
Premise
Schemas do not declare additionalProperties:false on main — this PR's own TestPrepareToolStampsClosedSchema asserts require.Nil(...) before the stamp. The change mints the contract and enforces it in one step rather than honoring an existing one, so "clients that work by accident" understates the break.
Suggested path
Default to warn; make reject opt-in (GORTEX_TOOL_ARG_GUARD=reject). That clears every item above and matches this issue's own ranking of option 2 as the first step.
If reject-by-default is wanted:
- Teach the three injectors to consult the published schema, or exempt a fixed response-shaping key set (
format,fields,max_bytes,max_tokens,cursor) insidewrapToolArgGuard. - Declare
max_charsonread_fileandcompactonsmart_context. - Add an integration test piping
buildToolCallFrame(...)output foraudit_health/verify_change/get_test_targetsthroughsrv.MCPServer().HandleMessage, assertingisError=false. The harness atarg_schema_guard_test.go:161already exists. - A lint test asserting no shipped tool reads an undeclared key prevents recurrence.
Correct as-is
The middleware seam is the right one: the guard survives lazy promotion (tool stamped before lazy.Register), covers control tools and both dispatch paths. reconcileToolParams (overlay.go:124) runs before the guard and deletes resolved alias keys, so typo recovery still works. checkToolGate runs first, so hidden tools leak no schema. The e2e test is not vacuous — the fixture's main.go really contains func helper(). Byte-ceiling re-base verified against main: agent had 8 bytes of slack (28192/28200) → 28743/29050; loc 20545 → 20922; core/full still under their pre-diet baselines.
Minor: warn appends only to Content, so it is invisible to structuredContent readers, and it also fires on error results. The off-vocabulary {off,0,false,none} diverges from the repo's {0,false,off,no} (parse_gate.go:97). The RawInputSchema branch is unreachable — 0/173 live tools use raw schemas — and its test comment about "the facade envelopes" is wrong; facades use structured schemas.
Both sides re-based the agent preset ceiling: main for the edit_file / write_file receipt + idempotency options, this branch for the zzet#597 additionalProperties:false stamp. Stack the two — measured on the merged tree and re-based with ~300-400 bytes of slack. The core pre-diet baseline moves for the same reason: main's growth had eaten most of the diet slack and the stamp is contract, not creep.
Review follow-up on the dispatch arg guard. Rejecting by default broke callers that work today: first-party surfaces inject undeclared keys into arbitrary tools (the CLI pins format into every legacy-surface frame, gortex call sets it for every non-facade tool, the HTTP bridge merges ?format=), and generic layers honor max_bytes / max_tokens / fields on every list-shaped response without any schema declaring them. - Default is now warn: the handler runs and the result carries an _ignored_options rider naming the unknown keys and the valid options. GORTEX_TOOL_ARG_GUARD=reject restores the hard refusal; the off vocabulary aligns with the repo's boolean-env idiom (0/false/off/no). - Response-shaping keys (format, fields, max_bytes, max_tokens, cursor) are exempt in both modes - dispatch demonstrably honors them, so flagging them is noise at best and a broken CLI at worst. - The rider skips error results and is mirrored into structuredContent when the result carries a structured map - Content alone is invisible to structured readers. - read_file declares its handler-honored max_chars; the subagent hook stops passing compact to smart_context (never a declared or honored option - the call worked only because unknown keys used to vanish). - The RawInputSchema branch is gone: 0 shipped tools use raw schemas and the zzet#597 stamp closes only structured ones, so it guarded a population of zero. - End-to-end pins: the CLI's format-pinned frames for audit_health / verify_change / get_test_targets survive guarded dispatch untouched, and the zzet#597 read_file(line_range) shape still refuses under reject while warning under the default.
|
Reworked in 26e9b62 (plus a merge of main in ff76686 that stacks the two ceiling re-bases). Took the suggested path: warn is the default, GORTEX_TOOL_ARG_GUARD=reject opts into the refusal. On top of that:
Not done: the lint test asserting no shipped tool reads an undeclared key. That needs a per-handler key inventory and felt like its own change; with the shaping set exempt and the two known gaps declared or removed, the known instances are covered. Can file it as a follow-up issue if you want it tracked. One thing this still does not change: facade dispatch invokes the pre-guard closure, so the guard remains inert on facade-v1 sessions where #597 was originally observed. The facade envelope validates its own operation vocabulary, but its options envelope stays open by design - if you want the guard inside the facade lane too, I would rather do that as a follow-up with its own tests than widen this PR further. Retitled the PR to match: it surfaces unknown options by default now, rejecting only on opt-in. |
zzet
left a comment
There was a problem hiding this comment.
Re-reviewed at 26e9b620. The rework lands: all four blockers from the last pass are fixed, and I re-verified each rather than taking them on trust. One new defect, and it's in the mechanism this PR exists for.
Verified clean this round — don't re-litigate
- The warn default can't be bypassed behind your back. mcp-go v0.58.0 does ship server-side
additionalPropertiesenforcement (WithInputSchemaValidation, plusWithStrictInputSchemaDefaultwhich does your stamp for you) — gortex enables neither, zero hits repo-wide. So stamping the schema closed genuinely cannot hard-reject while the guard says warn. - The CLI/HTTP shapes work.
formatexemption verified through the injectors; the CLI readsres.Content[0].Textonly, so an appended rider can't corrupt JSON decoding either. - All 13
callServerToolhook sites cross-checked against declared schemas — clean after thesubagent.gofix. (precompact.go:154get_symbol_history{compact}is fine, it's declared.) - Ordering is right.
reconcileToolParamsandcheckToolGateboth run before the guard, so typo recovery still works and the "valid options" gloss can't leak a gated tool's schema. Sanitize wraps the guard, so echoed key names pass the injection screen. - Byte re-base is honest. Measured main → PR: agent 28675→29313 (ceiling 29700), loc 20556→21020, core 94931→96168, full 249727→254357. That's ≈27 B/tool of stamp, not description creep.
- I ran the lint test you deferred, ad hoc: regex over every
req.Get*/Require*inside eachhandle*body, mapped to its tool via themcp.NewTool("x", …), s.handleXregistration. 0 closed tools read a key their own schema doesn't declare. That supports your claim that the known instances are covered and reject mode is safe now. One caveat it can't see: keys read in shared helpers.resolveScopeForRequestreadsrefon behalf of 8 tools that don't declare it (audit_health,find_clones,find_files,get_communities,get_processes,get_recent_changes,run_inspections,search_text) — butreconcileToolParamsalias-rewritesref→repo(levenshtein 2) before the guard sees it, so no rider fires. That rewrite is a pre-existing bug on main, not yours.
Blocker: the warn rider is silently eaten by the freshness and warmup decorators
The guard appends _ignored_options as a second text content block from inside the handler. decorateResultWithWarming (warmup_fastpath.go) and decorateResultWithFreshness / decorateListResultWithFreshness (freshness_rider.go) both end in rebuildTextResult, which builds a fresh single-block result from Content[0] and drops everything else — other blocks and StructuredContent. Those decorators run after the guard in wrapToolHandlerMode.
Reproduced directly, same call twice, only difference is whether the file drifted on disk:
NO-DRIFT rider=true
DRIFT freshness=true rider=false blocks=1
So read_file(line_range: …) against a file that changed on disk since indexing — the normal state mid-edit — returns the full un-windowed response with no signal at all. That is exactly #597, on this PR's poster-child tool. Same for get_symbol_source, get_symbol, get_file_summary, get_editing_context, and the list tools search_symbols / find_usages / smart_context / get_callers, plus every graph tool during daemon warmup.
The fix is small — the repo already has a seam that survives: maybeAttachMomentumNote (internal/mcp/overlay.go:203) appends after the rebuilds. Either move the rider there, or embed _ignored_options into the JSON body the way freshness does. Worth a test that drifts a file and asserts the rider survives the decorator chain; nothing covers that today, which is why this passed green.
Minor, non-blocking
GORTEX_TOOL_ARG_GUARDis documented nowhere.docs/mcp.mddocumentsGORTEX_TOOLS,GORTEX_LAZY_TOOLSandGORTEX_OVERLAY_IDLE_TTLinline; a switch that changes dispatch semantics should join them.- The stamp publishes a contract slightly stricter than reality. Among the 153 closed tools,
formatis undeclared on 55,max_byteson 105,max_tokenson 141,cursoron 148,fieldson 149 — and all five are still accepted. Sharpest instance:read_file's own description says "Composable withformat:"gcx"" while its schema now forbidsformat. Nothing breaks at runtime, but models and schema-aware clients are being steered away from documented, working options. - The unknown-key list echoed into the rider/error is uncapped in both count and length.
On scope
Agreed on the facade lane as a follow-up, and thanks for stating plainly that the guard stays inert on facade-v1 — that's the default surface for named MCP clients, so it's worth being explicit that merging this does not close #597 where it was observed. Same for the declared-vs-read lint test: file it, and feel free to seed it with the sweep above, including the shared-helper caveat that makes resolveScope's ref invisible to a body-only scan.
Fix the rider ordering and the docs line and I'm happy to merge.
|
Both required items are in, plus one of the minors and the two follow-up filings. Rider ordering — fixed at the seam you pointed atThe guard no longer appends mid-chain. It records the warn verdict in a request-scoped slot ( Two properties from your verified-clean list survive by construction:
Both rebuild seams are pinned by tests that failed red against the pre-fix code with exactly your fingerprint (decorated block present, rider gone, one content block):
Docs
Minor 3 — taken hereThe echoed unknown-key list is now capped in both the rider and the reject error: at most 5 keys, each cut at 80 runes, with a Scope follow-ups — filed
Full |
Addresses #597.
Problem
Request decoding reads the keys it knows and unknown keys simply vanish. The measured shape from the issue: an agent passed
line_range: [120, 160]toread_filethree times — a plausible guess for an option that doesn't exist — and got the full file back each time, silently. Maximum token cost on the heaviest tool in the surface, for a call that explicitly asked for a 40-line window, with no signal to self-correct.Why reject, not the rider I proposed first
The issue ranked a warning rider (option 2) as the natural first step. Running agents against my own index changed my mind: the rider arrives attached to the full-cost response — the tokens are already spent, and the correction only helps the next call. An immediate error costs a few dozen tokens, names the bad key, and the agent fixes the very next call; the expensive wrong answer is never produced. So this PR makes dispatch honor the schema's own contract by default, with the rider and the old behavior both one env var away:
read_file does not accept option(s): line_range; valid options: …GORTEX_TOOL_ARG_GUARD=warn— run the handler, append an_ignored_optionsrider (option 2 as described in the issue)GORTEX_TOOL_ARG_GUARD=off— pre-guard behaviorChange
prepareToolcloses any structured schema that never took a position (additionalPropertiesunset →false), sotools/listpublishes exactly what dispatch enforces. Tools that opt into open args (AdditionalProperties(true)) keep them — enforcement follows the schema in both directions, raw schemas as authored.wrapToolArgGuardenforces at the same seam as the existing overlay/telemetry middleware, so both the daemon-dispatched path and the in-process HTTP path get it, and lazy-promoted tools inherit it.optionsenvelopes are open by design. Per-operation key enforcement inside those envelopes would be the follow-up — that one's a design conversation, not a patch.tools/listbyte ceilings re-based (+~27 bytes per tool for the published stamp), following the constants' own re-base convention.Tests
Nine pins: rejection happens before the handler runs (the handler-never-ran assertion is the point of the whole change), valid/nil args pass, explicit-open structured and raw schemas stay unenforced, closed raw schemas reject typo'd keys, warn mode runs the handler and names what it ignored, off mode restores old behavior,
prepareToolpublishes the closed schema — and the issue's exact shape end to end through real MCP frames:read_file(line_range: [120,160])refuses before the read, the corrected call works. Full suite matches my Windows baseline.