feat: max_chars result caps with a token receipt on every routed read

[brainsparker]

What this does

Frugal prices every call in dollars. The other cost of a tool call is the tokens its result occupies in the model's context for the rest of the session, and until now that cost had no receipt and no knob. This PR adds both.

• A max_chars argument on frugal__extract, frugal__browse, and frugal__execute caps the page content returned per call (markdown + text + html, one shared budget, in that priority order so the readable rendering survives and raw HTML is the first thing dropped).
• An optional limits.max_chars in models.yaml sets a server-wide default. A per-call max_chars overrides it in either direction. Zero or absent means no cap, which is also the default: an existing config returns results byte-for-byte as before.
• Every response now carries a size footer: chars_returned, chars_total, est_tokens, and truncated: true when the two counts differ. frugal__search reports est_tokens for the result list (search hits are measured, never cut; max_results is the size knob there).
• The cut is rune-safe, backs off to the nearest word boundary, and appends a visible marker to the shortened field: [frugal: output truncated to 1490 of 157476 chars; pass a larger max_chars to see more]. Structured output carries the same numbers as fields, but clients that flatten results to text would otherwise lose the signal entirely.

limits:
  max_chars: 40000   # about 10k tokens, where Claude Code starts warning about MCP output size

Captured from a live zero-key run against a real page:

frugal__extract {"url": "https://en.wikipedia.org/wiki/Model_Context_Protocol", "max_chars": 1500}

  result › {
    "provider_used": "goreadability",
    "cost_usd": 0,
    "chars_returned": 1490,
    "chars_total": 157476,
    "truncated": true,
    "est_tokens": 373,
    "markdown": "...[frugal: output truncated to 1490 of 157476 chars; pass a larger max_chars to see more]"
  }

Without the cap that page is roughly 39,000 tokens. Claude Code's default MCP ceiling is 25,000, so the call would have failed outright.

Why this, why now

Clients enforce result-size ceilings blind, and the failure modes are documented:

• Claude Code rejects any MCP tool result over MAX_MCP_OUTPUT_TOKENS (default 25,000 tokens) with a plain error, and separately spills any result over 50,000 characters to a temp file with a 2 KB head preview. There is no env var for the second layer. Anthropic's own SDK tests document both layers: https://github.com/anthropics/claude-agent-sdk-python/blob/e41cbdd4/tests/test_mcp_large_output.py
• Open issue Tool result size cap (~25K tokens) blocks round-tripping large MCP tool payloads anthropics/claude-code#45770 describes the correctness hazard: when a large structured result is head-truncated, the preview looks complete enough that the model proceeds on the visible prefix and fabricates the missing fields, because nothing signals that load-bearing content was cut. The two mitigations the issue asks for are exactly what this PR does server-side: make truncation unmistakable, and put the cut where the tool author chooses rather than at a blind byte offset. Tool result size cap (~25K tokens) blocks round-tripping large MCP tool payloads anthropics/claude-code#45770
• Token bloat from MCP payloads is the loudest complaint in the ecosystem this month. Aug 22 measurements put one MCP-heavy Claude Code conversation at about 120,000 tokens of overhead before real output: https://dev.to/mcptokensaver/real-token-cost-of-mcp-91k-tokens-of-json-4goe. Aug 26 follow-up on the same theme: https://dev.to/mech_app_ai/mcps-token-overhead-why-agent-tool-protocols-burn-4-32x-more-tokens-than-cli-and-how-to-fix-it-20dn
• Firecrawl's guide on Claude Code web research makes the session-level point: a page fetched early stays in context for every later turn, so shaping what lands in context is the highest-leverage cost move for research agents. https://www.firecrawl.dev/blog/claude-code-web-research-tokens

Frugal already owns the interception point for every search, extract, and browse result and already ships the cost receipt. Adding the size receipt and the size knob is the same product promise applied to the second cost. It is independent of the open exact-match cache PR (#89). One note for whichever lands second: the cap should apply after the cache read, so that a hit with a larger max_chars still returns more; I kept the cap at the very end of each handler to make that rebase trivial.

Runner-up candidates considered today and passed on:

1. MCP-Profiles (you-aware): read package.json / go.mod / pyproject.toml at startup and merge pinned dependency versions into compiled queries, closing the "wrong framework version" gap Context7 markets against. Strong fit, but the sandbox could not reach the npm registry today, so it could not be built or tested. Held for a day when it can be verified.
2. frugal: a Parallel or Exa search provider as a new cheap rung, given 2026 price moves (Exa up 40 percent to $7 per 1k, Brave dropped its free plan). Real, but the pricing claims needed more verification than fit in one day, and a cap that works for every provider beat one more provider.
3. PromptLens and you.md: both have multiple unreviewed daily PRs open (13 and 3), including near-duplicates. Adding more before those are triaged is review debt, not value.

Implementation notes

• New internal/limit package: Cap(maxChars, fields...) applies one budget across fields in order, rune-safe, with a 256-character whitespace back-off and a marker on the shortened field; EstTokens is ceil(chars / 4) (a planning figure, documented as such); Count sums rune lengths. No dependencies.
• internal/config: new optional limits: section with max_chars. Validation rejects negative values with a clear message; zero is the documented "no cap" spelling. KnownFields still enforced.
• internal/mcp/tools: one new ToolOption (WithMaxChars) consumed by all four registrations, effectiveMaxChars resolves per-call over default over unlimited. Extract caps markdown, text, html; browse caps text, html; execute applies the same cap to extract and browse intents (including the extract-to-browse fall-forward path) and measures search intents without cutting. Metadata fields (title, byline, links) and the routing receipt are never touched.
• cmd/frugal/mcp.go: reads cfg.Limits, logs the default when set, passes it through.
• Docs: README section "Result size caps and the token receipt" placed after the spend-cap section, and a commented limits: block in config/models.yaml.
• No em dashes in any shipped text.

Test status

• 9 new unit tests in internal/limit (rounding, rune safety, shared budget order, whitespace back-off, hard cut fallback, marker text, nil and empty fields, zero means unlimited).
• 11 new integration tests in internal/mcp/tools/limits_test.go driving real in-memory MCP client sessions: no-cap path is byte-for-byte unchanged, per-call cap truncates and reports, configured default applies and per-call overrides it both ways, markdown preferred over html, negative max_chars errors, browse caps text before html, execute honors the cap on extract intents and on the fall-forward render, search intents are measured but never truncated, frugal__search reports est_tokens.
• 3 new config tests (round trip, negative rejected, absent and zero both uncapped).
• Full suite: gofmt -l clean, go build ./..., go vet ./..., go test ./... all green locally on Go 1.25.1.
• Live stdio smoke test against the built binary: tools/list shows max_chars on extract and execute (not search), and the Wikipedia extract above returned the expected footer and marker.

[cloudflare-workers-and-pages]

Deploying with    Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	frugal	bd0e4c3	Commit Preview URL Branch Preview URL	Sep 03 2026, 01:18 PM