feat: opt-in exact-match result cache for search and extract (zero-cost repeat calls)

[brainsparker]

What this does

Adds an opt-in, exact-match result cache to the routed read tools. When enabled, a repeated identical call inside the TTL is answered from process memory instead of a provider, at zero cost:

cache:
  enabled: true
  search_ttl: 5m      # default 5m; "0" turns search caching off
  extract_ttl: 15m    # default 15m; "0" turns extract caching off
  max_entries: 512    # LRU eviction past this bound

Behavior:

• Hits are labeled, never silent: cached: true, cache_age_ms, cost_usd: 0, with provider_used still naming the provider that produced the original result.
• frugal__execute shares entries with the direct tools. frugal__execute("search python docs") is a hit after frugal__search("python docs") and vice versa. An explicit priority: cheap or premium bypasses the cache (read and write): the caller asked for a specific routing outcome, and a memoized answer from a different chain would misreport it.
• Exact-match keys cover everything that changes what a provider would return: query or URL, max_results, freshness, formats (as a case-insensitive set), and any provider pin. Empty pin and "auto" share a key.
• frugal__browse is never cached. Rendering a page is exactly the case where the caller wants the live DOM.
• In-memory only, per process, LRU-bounded. Cached provider payloads never touch disk and never outlive the server. Off by default: absent config keeps the historical call-every-time behavior exactly.

Why this, why now

Agents repeat themselves. Retry loops, sibling subagents issuing the same query, and follow-up turns re-fetching the same page are the documented waste pattern of 2026 agent stacks:

• ToolCaching (arXiv, Jan 2026) measures redundant tool calls in LLM workflows as a first-order latency and cost driver and proposes caching as the fix: https://arxiv.org/html/2601.15335v1
• The MCP ecosystem is converging on the same conclusion from the ops side: exact-duplicate tool-call dedup and memoization keep appearing as standalone middleware (tool-call-cache and session-scoped dedup writeups, May 2026: https://dev.to/mukundakatta/stop-paying-for-the-same-tool-call-twice-memoize-llm-tool-calls-with-tool-call-cache-56kp), in MCP proxy playbooks (https://mcptrail.com/blog/mcp-token-optimization/), and in an MCP idempotency SEP reference implementation (Aug 2026: https://github.com/abluva/mcp-request-idempotency-reference)
• Practitioner guides now treat tool-result caching as the highest-value cache target for MCP servers specifically (Feb 2026: https://fast.io/resources/mcp-server-caching/)

frugal already owns the interception point (every search / extract / browse call routes through it), already prices every call, and already ships the receipt that proves savings. A cache that returns cost_usd: 0 on hits is the most on-brand feature this repo can ship. The roadmap lists a Phase 3 semantic cache; this lands the deterministic exact-match layer that work builds on, and the package comment in internal/mcp/tools/search.go has reserved internal/cache for this since the tool layer landed.

Runner-up candidates considered today: an MCP 2026-07-28 spec migration for frugal (timely after the stateless-core release, but a larger lift gated on go-sdk support, and better done deliberately), and further PromptLens / you.md features (both repos have multiple unreviewed daily PRs stacked up, including near-duplicates in PromptLens; adding more before those are triaged creates review debt, not value).

Implementation notes

• New internal/cache package: bounded TTL + LRU store with a nil-safe API (mirrors the routing.Guard convention, so call sites need no conditionals), an injectable clock for tests, and hit / miss / saved-USD counters exposed via Snapshot() for a future frugal stats line.
• Key canonicalization lives next to the store: cache.SearchKey and cache.ExtractKey normalize pins, trim queries, and treat formats as a case-insensitive set. Query case is preserved: exact match means exact.
• Tool wiring is one new ToolOption (WithResultCache) consumed by search, extract, and execute registration. Execute stores and reads through the same keys inside dispatchIntent, and only on the clean single-capability paths: the extract-to-browse fall-forward result is never cached.
• Config: new optional cache: section with validation (negative max_entries fails the load; invalid TTLs warn at wiring time and fall back to defaults, same as routing.cooldown).
• A denied or budget-blocked provider can never leak through the cache: policy and caps are fixed at startup, the cache is per-process, and blocked pins error before dispatch, so no such entry can exist.
• Docs: README section ("Result cache") and a commented block in config/models.yaml.

Test status

• 13 new tests: 9 in internal/cache (TTL expiry, LRU eviction, age reporting, nil-safety, key canonicalization), 5 integration tests in internal/mcp/tools driving real in-memory MCP client sessions (second call hits, distinct args miss, disabled cache never labels, execute shares entries with search in both directions, premium priority bypasses), 3 config parse tests.
• Full suite: go build ./..., go vet ./..., and go test ./... all green locally on Go 1.25.

[cloudflare-workers-and-pages]

Deploying with    Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	frugal	e2e64e4	Commit Preview URL Branch Preview URL	Aug 29 2026, 01:14 PM