[Feature] Prompt / Context Caching

[akramcodez]

Description

Implement prompt/context caching using provider-specific options (e.g., Anthropic's cacheControl breakpoints or OpenAI's automatic prefix caching). Since the system prompt, tool schemas, and conversation history are resent every turn, caching these prefixes will drastically cut latency and cost.

Use Case

For multi-turn sessions with large system prompts and tool definitions, users currently pay token costs and latency penalties on every turn for static context.

Proposed Solution

Modify source/ai-sdk-client/chat/provider-options.ts to set cache-control hints. Mark stable prefixes (system prompt + tool definitions) as cacheable.

Alternatives Considered

None. This is a native feature of newer model APIs that just needs to be wired up.

Additional Context

• I have searched existing issues to ensure this is not a duplicate
• This feature aligns with the project's goals (local-first AI assistance)

Implementation Notes

This is the single highest-leverage change identified for reducing multi-turn latency and cost.

[addyCooks]

Hi @akramcodez!

Can you assign this to me?
What I'm thinking to do:

1. provider-options.ts is the wrong hook it builds request-level options; Anthropic's cacheControl is per-message, so this belongs in message-converter.ts.
2. system: has to become a message a bare string can't carry providerOptions.
3. Max 4 breakpoints, and /compact invalidates everything downstream needs a heuristic + config toggle, not always-on.
4. Cost reporting can be the real risk response-usage.ts has no cached-token concept,
   so /usage will overstate cost on every hit. cache_read/cache_write are already in models-types.ts:16, just dropped by getModelPricing.
5. Anthropic-only OpenAI/OpenRouter prefix-cache automatically,
   and it's a no-op for local models.