feat(ai): default claude-* models to automatic Anthropic prompt caching

artifacts/issue-3670-anthropic-cache-eval.json

@@ -6,8 +6,8 @@
 	"source": {
 		"url": "https://platform.claude.com/docs/en/build-with-claude/prompt-caching",
 		"retrievedAt": "2026-07-18",
-		"providerSourceBlobOid": "ca40efcc01da0ebbc78f018563a8a4474d77ba36",
-		"providerSourceSha256": "b3173948f5982c97b41790f53fbc396d514e5ce36cfa316453447527aa0236e8",
+		"providerSourceBlobOid": "bc15958b9b80add7af990abd0e82d0450640a931",
+		"providerSourceSha256": "3d926409bf0a7b4ec1d7222a4c402f18b3ee5c01a5f01c6d84d8894bda16e963",
 		"inputFixtureSha256": "b1ca8d3183ca168140774f848ed414252178f7c5788d61243069862ae59c129b"
 	},
 	"derivationCommands": [

docs/models.md

@@ -284,7 +284,7 @@ providers:
 - `auth`: `apiKey` (default), `none`, or `oauth`; for `models.yml` custom models, `oauth` is accepted by schema but does not waive the `apiKey` requirement
 - `models.yml` is strict: unknown provider/model keys fail validation before provider dispatch, so stale keys such as `requestTransform` or `wireModelId` only work where this document lists them.
 - `discovery.type`: `ollama`, `llama.cpp`, `lm-studio`, or `openai-models-list`
-- `cacheRetention`: `none`, `short`, or `long`; request-time options win over model/modelOverride values, then provider values, then `GJC_CACHE_RETENTION`, then the runtime default. The runtime default is `short` for most providers, but the Anthropic provider defaults to `long` (`ttl: "1h"`) because the ~5m default is too fragile for long-running subagent workflows. The 1h marker is only emitted on the canonical Anthropic API (`api.anthropic.com`) for models advertising `supportsLongCacheRetention`; proxies, gateways, and incapable models fall back to the default ephemeral (~5m) breakpoint. For OpenAI Responses, this controls `prompt_cache_retention` only; it does not disable `prompt_cache_key` when a stable session id exists.
+- `cacheRetention`: `none`, `short`, or `long`; request-time options win over model/modelOverride values, then provider values, then `GJC_CACHE_RETENTION`, then the runtime default. The runtime default is `short` for most providers, but the Anthropic provider defaults to `long` because the ~5m cache is fragile for long-running subagent workflows. Canonical Anthropic models emit `ttl: "1h"` when long retention is supported. Claude-family models on non-canonical Anthropic-compatible endpoints now use top-level automatic caching by default but omit `ttl` (the provider's ~5m default) unless `compat.supportsLongCacheRetention: true` explicitly opts the endpoint into 1-hour retention. For OpenAI Responses, this controls `prompt_cache_retention` only; it does not disable `prompt_cache_key` when a stable session id exists.
 
 ## OpenAI-compatible proxy configuration
 
@@ -803,7 +803,32 @@ Provider-level `compat` is the baseline; per-model `compat` is deep-merged on to
 
 ### Anthropic compatibility (`anthropic-messages`)
 
-For `anthropic-messages` models the runtime uses a separate `AnthropicCompat` shape (`packages/ai/src/types.ts`). The `models.yml` schema currently exposes only the strict-tools opt-out as a top-level provider field (see below); the remaining Anthropic-side knobs (`disableAdaptiveThinking`, `supportsEagerToolInputStreaming`, `supportsLongCacheRetention`) are set by built-in catalog metadata and are not user-configurable from `models.yml`.
+For `anthropic-messages` models, `compat.promptCacheMode` and `compat.supportsLongCacheRetention` are configurable at provider, model, and `modelOverrides` levels. Provider-level `compat` is the baseline; model and override values merge on top.
+
+Prompt-cache modes:
+
+- `automatic` — emit one top-level `cache_control` marker and let the Anthropic-compatible endpoint advance the breakpoint as the conversation grows.
+- `explicit` — emit block-level breakpoints instead. Use this for endpoints that reject top-level `cache_control` but support Anthropic's explicit content-block markers.
+- `none` — emit no generated Anthropic cache controls. Per-request or configured `cacheRetention: none` also disables generated caching.
+
+Without an explicit mode, canonical Anthropic endpoints and Claude-family model ids default to `automatic`; unknown non-Claude compatible endpoints default to `none`. Non-canonical endpoints get the default ~5m lifetime unless they opt into `supportsLongCacheRetention: true`.
+
+```yaml
+providers:
+  corp-anthropic:
+    baseUrl: https://proxy.example.com/anthropic
+    apiKeyEnv: CORP_ANTHROPIC_API_KEY
+    api: anthropic-messages
+    compat:
+      promptCacheMode: explicit
+      supportsLongCacheRetention: false
+    models:
+      - id: claude-sonnet-4-5
+        contextWindow: 200000
+        maxTokens: 8192
+```
+
+Other Anthropic-side compatibility knobs such as `disableAdaptiveThinking` and `supportsEagerToolInputStreaming` remain built-in catalog metadata rather than `models.yml` fields. `disableStrictTools` stays a provider-level setting (below).
 
 ### Strict tool schemas (`disableStrictTools`)
 

packages/ai/CHANGELOG.md

@@ -2,12 +2,17 @@
 
 ## [Unreleased]
 
+### Changed
+
+- Anthropic prompt caching now defaults to top-level automatic caching (`cache_control: { type: "ephemeral" }`) for every Claude-family model, including through non-canonical Anthropic-compatible gateways (Cloudflare AI Gateway, GitHub Copilot, GitLab Duo, Vercel AI Gateway, zenmux, etc.), instead of only `api.anthropic.com`. Non-Claude models on unknown compatible endpoints keep the previous no-cache default; `compat.promptCacheMode: "none"`, `compat.promptCacheMode: "explicit"`, and configured or per-request `cacheRetention: "none"` still opt out. Non-canonical Claude models get the default ~5m cache lifetime unless the endpoint sets `compat.supportsLongCacheRetention: true`.
+
 ### Fixed
 
 - `todo_write` raw argument rejections now carry bounded, authority-controlled correction codes for each rejected shape: unknown root keys, unknown operation-entry keys, done/drop entries missing a task or phase target, and unknown init list-entry keys. Each code maps to a fixed correction message naming the accepted shape (never echoing the offending input), so invalid calls surface specific guidance while valid payloads keep the existing passthrough/coercion path (#3916).
 - Anthropic Sonnet 5 now exposes Anthropic's real `xhigh` and `max` thinking efforts on the Messages API (`minimal`/`low`/`medium`/`high`/`xhigh`/`max`), matching official support. The previous generic `kind === opus` gate excluded it from the full preset range; the capability predicate is now an explicit version-scoped list (Opus 4.7+, Sonnet 5+), so older Sonnet generations and Bedrock Converse routes stay fail-closed at their previously advertised levels (issue #3913).
 - Alibaba Token Plan now exposes Qwen 3.8 Max under the provider-supported `qwen3.8-max` wire id instead of the rejected `qwen-3.8-max` spelling; catalog regeneration canonicalizes a legacy discovered alias rather than retaining a broken duplicate (#3909).
 - Canonicalized first-class MiniMax M3 catalog ids (issue #3896). The bundled catalog previously shipped stale lowercase `minimax-m3` duplicates (512K) next to the canonical `MiniMax-M3` (1M) on all four first-class MiniMax providers, plus a non-official `minimax-v3` entry under `minimax-code`. The lowercase `minimax-m3` entries and `minimax-v3` are removed; `MiniMax-M3` is the single canonical first-class id (the regen-safe 1M pin in `applyGeneratedModelPolicy` now keys on `MiniMax-M3` / `MiniMax-M3[1m]` instead of the removed lowercase id), `DEFAULT_MODEL_PER_PROVIDER` points at `MiniMax-M3`, and the official Anthropic Token Plan id `MiniMax-M3[1m]` is first-class on the `minimax` / `minimax-cn` Anthropic routes with 1M context semantics. Unrelated catalog providers keep their own `minimax-m3` contracts.
+- Anthropic cache-control resolution now falls back to `model.cacheRetention` at the provider boundary, preserving configured retention and request-over-model precedence through special dispatch wrappers such as GitLab Duo. A configured `cacheRetention: "none"` can no longer be dropped and replaced by the new automatic Claude-family cache marker.
 ## [0.12.12] - 2026-08-05
 
 ### Fixed

packages/ai/src/providers/anthropic.ts

@@ -447,12 +447,21 @@ function dropAnthropicStrictTools(params: MessageCreateParamsStreaming): void {
 	}
 }
 
+function isClaudeFamilyModel(model: Model<"anthropic-messages">): boolean {
+	// Classify the same identifier the request body serializes (`params.model =
+	// model.id` in buildParams); a differing `wireModelId` is not dispatched by
+	// this transport, so it must not drive the cache decision either.
+	const id = model.id;
+	const shortId = id.includes("/") ? id.slice(id.lastIndexOf("/") + 1) : id;
+	return shortId.toLowerCase().startsWith("claude-");
+}
+
 function getCacheControl(
 	model: Model<"anthropic-messages">,
 	baseUrl: string,
 	cacheRetention?: CacheRetention,
 ): { mode: AnthropicCacheMode; cacheControl?: AnthropicCacheControl } {
-	const retention = resolveCacheRetention(cacheRetention, "long");
+	const retention = resolveCacheRetention(cacheRetention ?? model.cacheRetention, "long");
 	if (retention === "none") return { mode: "none" };
 
 	const isCanonicalApi = isAnthropicApiBaseUrl(baseUrl);
@@ -462,7 +471,7 @@ function getCacheControl(
 			? "none"
 			: promptCacheMode === "explicit"
 				? "explicit"
-				: isCanonicalApi
+				: promptCacheMode === "automatic" || isCanonicalApi || isClaudeFamilyModel(model)
 					? "automatic"
 					: "none";
 	if (mode === "none") return { mode };

packages/ai/src/types.ts

@@ -951,8 +951,10 @@ export interface AnthropicCompat extends ToolChoiceCompat {
 	supportsLongCacheRetention?: boolean;
 	/**
 	 * Prompt-cache transport accepted by this Anthropic-compatible endpoint.
-	 * Canonical Anthropic defaults to `"automatic"`; noncanonical endpoints default
-	 * to `"none"` and must explicitly opt into generated `"explicit"` markers.
+	 * Canonical Anthropic and Claude-family models default to `"automatic"`;
+	 * noncanonical non-Claude endpoints default to `"none"`. Set `"automatic"` to
+	 * opt an otherwise unknown compatible endpoint into top-level caching, `"none"`
+	 * to opt out, or `"explicit"` for endpoints that require block-level markers.
 	 */
 	promptCacheMode?: "none" | "explicit" | "automatic";
 }

packages/ai/test/anthropic-cache.test.ts

@@ -1,7 +1,8 @@
 import { describe, expect, it } from "bun:test";
 import type { MessageCreateParamsStreaming } from "@anthropic-ai/sdk/resources/messages";
 import { normalizeCacheControlTtlOrdering, streamAnthropic } from "@gajae-code/ai/providers/anthropic";
-import type { Context, Model, TJsonSchema } from "@gajae-code/ai/types";
+import { clearGitLabDuoDirectAccessCache, streamGitLabDuo } from "@gajae-code/ai/providers/gitlab-duo";
+import type { CacheRetention, Context, Model, TJsonSchema } from "@gajae-code/ai/types";
 
 const canonicalModel: Model<"anthropic-messages"> = {
 	id: "claude-sonnet-4-5",
@@ -43,12 +44,14 @@ function capturePayload(
 	model: Model<"anthropic-messages">,
 	input: Context,
 	onPayload?: (payload: Payload) => Payload | undefined,
+	cacheRetention?: CacheRetention,
 ): Promise<Payload> {
 	const { promise, resolve } = Promise.withResolvers<Payload>();
 	streamAnthropic(model, input, {
 		apiKey: "sk-ant-api-test",
 		isOAuth: false,
 		signal: abortedSignal(),
+		cacheRetention,
 		onPayload: payload => {
 			const replacement = onPayload?.(payload as Payload);
 			resolve((replacement ?? payload) as Payload);
@@ -57,6 +60,32 @@ function capturePayload(
 	});
 	return promise;
 }
+function captureGitLabPayload(model: Model<"anthropic-messages">, cacheRetention?: CacheRetention): Promise<Payload> {
+	clearGitLabDuoDirectAccessCache();
+	const { promise, resolve } = Promise.withResolvers<Payload>();
+	streamGitLabDuo(model, context(), {
+		apiKey: "glpat-test",
+		signal: abortedSignal(),
+		cacheRetention,
+		fetch: async input => {
+			const url = input instanceof Request ? input.url : String(input);
+			if (url === "https://gitlab.com/api/v4/ai/third_party_agents/direct_access") {
+				return new Response(
+					JSON.stringify({ token: "direct-token", headers: { "x-gitlab-instance-id": "test" } }),
+					{
+						status: 200,
+						headers: { "content-type": "application/json" },
+					},
+				);
+			}
+			throw new Error(`Unexpected GitLab Duo fetch: ${url}`);
+		},
+		onPayload: payload => {
+			resolve(payload as Payload);
+		},
+	});
+	return promise;
+}
 
 function cacheParams(overrides: Partial<Payload> = {}): Payload {
 	return {
@@ -98,21 +127,128 @@ describe("Anthropic prompt caching", () => {
 		compat: { promptCacheMode: "explicit" },
 	};
 
-	it("defaults canonical Anthropic to automatic and requires compatible endpoints to opt into explicit caching", async () => {
-		const [canonical, compatible, explicit] = await Promise.all([
+	it("defaults canonical Anthropic and Claude-family models to automatic caching", async () => {
+		const nonClaudeModel: Model<"anthropic-messages"> = {
+			...canonicalModel,
+			id: "custom-compatible-model",
+			name: "Custom compatible model",
+			baseUrl: "https://proxy.example.test/anthropic",
+		};
+		const [canonical, proxiedClaude, explicit, nonClaude] = await Promise.all([
 			capturePayload(canonicalModel, context()),
 			capturePayload({ ...canonicalModel, baseUrl: "https://proxy.example.test/anthropic" }, context()),
 			capturePayload(explicitCompatibleModel, context()),
+			capturePayload(nonClaudeModel, context()),
 		]);
 
 		expect(canonical.cache_control).toEqual({ type: "ephemeral", ttl: "1h" });
-		expect(compatible.cache_control).toBeUndefined();
+		// Claude-family models behind non-canonical Anthropic-compatible gateways
+		// get top-level automatic caching; without explicit long-retention support
+		// they fall back to the default ~5m breakpoint (no ttl).
+		expect(proxiedClaude.cache_control).toEqual({ type: "ephemeral" });
 		expect(explicit.cache_control).toBeUndefined();
 		expect(explicit.tools?.every(tool => !(tool as { cache_control?: CacheControl }).cache_control)).toBe(true);
 		expect(!Array.isArray(explicit.system) || explicit.system.every(block => !block.cache_control)).toBe(true);
 		expect((explicit.messages.at(-1)?.content as Array<{ cache_control?: CacheControl }>)[0]?.cache_control).toEqual({
 			type: "ephemeral",
 		});
+		// Non-Claude models on unknown compatible endpoints still get no generated caching.
+		expect(nonClaude.cache_control).toBeUndefined();
+	});
+	it("classifies the dispatched model id and honors every cache opt-out", async () => {
+		const proxyUrl = "https://proxy.example.test/anthropic";
+		const cases: Array<{
+			name: string;
+			model: Model<"anthropic-messages">;
+			options?: { cacheRetention?: "none" | "short" | "long" };
+			expected: CacheControl | undefined;
+		}> = [
+			{
+				name: "prefixed claude id on non-canonical gateway",
+				model: { ...canonicalModel, id: "anthropic/claude-sonnet-4-5", baseUrl: proxyUrl },
+				expected: { type: "ephemeral" },
+			},
+			{
+				name: "uppercase claude id on non-canonical gateway",
+				model: { ...canonicalModel, id: "CLAUDE-OPUS-5", baseUrl: proxyUrl },
+				expected: { type: "ephemeral" },
+			},
+			{
+				name: "non-canonical claude with explicit long retention opt-in",
+				model: { ...canonicalModel, baseUrl: proxyUrl, compat: { supportsLongCacheRetention: true } },
+				expected: { type: "ephemeral", ttl: "1h" },
+			},
+			{
+				name: "promptCacheMode none disables automatic caching",
+				model: { ...canonicalModel, baseUrl: proxyUrl, compat: { promptCacheMode: "none" } },
+				expected: undefined,
+			},
+			{
+				name: "per-request cacheRetention none disables caching on canonical",
+				model: canonicalModel,
+				options: { cacheRetention: "none" },
+				expected: undefined,
+			},
+			{
+				name: "id containing -claude- but not starting claude- is not cached",
+				model: { ...canonicalModel, id: "my-claude-helper", baseUrl: proxyUrl },
+				expected: undefined,
+			},
+			{
+				name: "promptCacheMode automatic opts a non-Claude endpoint into top-level caching",
+				model: {
+					...canonicalModel,
+					id: "custom-compatible-model",
+					baseUrl: proxyUrl,
+					compat: { promptCacheMode: "automatic" },
+				},
+				expected: { type: "ephemeral" },
+			},
+			{
+				name: "wireModelId override does not drive the decision; dispatched id governs",
+				model: {
+					...canonicalModel,
+					id: "local-alias",
+					wireModelId: "claude-sonnet-4-5",
+					baseUrl: proxyUrl,
+				},
+				expected: undefined,
+			},
+			{
+				name: "wireModelId override to a non-claude wire id still follows dispatched id",
+				model: {
+					...canonicalModel,
+					id: "claude-sonnet-4-5",
+					wireModelId: "local-alias",
+					baseUrl: proxyUrl,
+				},
+				expected: { type: "ephemeral" },
+			},
+		];
+
+		for (const { name, model, options, expected } of cases) {
+			const payload = await capturePayload(model, context(), undefined, options?.cacheRetention);
+			expect(payload.model, name).toBe(model.id);
+			expect(cacheControls(payload), name).toEqual(expected ? [expected] : []);
+		}
+	});
+	it("preserves configured cache retention through GitLab Duo and lets request options win", async () => {
+		const gitlabModel: Model<"anthropic-messages"> = {
+			...canonicalModel,
+			id: "duo-chat-sonnet-4-6",
+			name: "Duo Chat Sonnet 4.6",
+			provider: "gitlab-duo",
+			baseUrl: "https://cloud.gitlab.com/ai/v1/proxy/anthropic/",
+			cacheRetention: "none",
+		};
+
+		const configuredNone = await captureGitLabPayload(gitlabModel);
+		const requestOverride = await captureGitLabPayload(gitlabModel, "short");
+
+		expect(configuredNone.model).toBe("claude-sonnet-4-6");
+		expect(cacheControls(configuredNone)).toEqual([]);
+		expect(requestOverride.model).toBe("claude-sonnet-4-6");
+		expect(cacheControls(requestOverride)).toEqual([{ type: "ephemeral" }]);
 	});
 
 	it("counts top-level automatic and caller controls together without mutating a callback replacement", async () => {

packages/ai/test/anthropic-stream-envelope.test.ts

@@ -1127,6 +1127,12 @@ describe("anthropic stream envelope handling", () => {
 			model,
 			{ ...model, compat: { supportsLongCacheRetention: false } },
 			{ ...model, baseUrl: "https://proxy.example.com/anthropic" },
+			{
+				...model,
+				id: "custom-compatible-model",
+				name: "Custom compatible model",
+				baseUrl: "https://proxy.example.com/anthropic",
+			},
 		]) {
 			const stream = streamAnthropic(testModel, context, {
 				apiKey: "sk-ant-test",
@@ -1143,7 +1149,11 @@ describe("anthropic stream envelope handling", () => {
 		);
 		expect(cacheControls[0]).toEqual({ type: "ephemeral", ttl: "1h" });
 		expect(cacheControls[1]).toEqual({ type: "ephemeral" });
-		expect(cacheControls[2]).toBeUndefined();
+		// Claude-family models through Anthropic-compatible gateways get automatic
+		// caching; without explicit long-retention support the default ~5m breakpoint applies.
+		expect(cacheControls[2]).toEqual({ type: "ephemeral" });
+		// Non-Claude models on unknown compatible endpoints receive no generated caching.
+		expect(cacheControls[3]).toBeUndefined();
 	});
 
 	it("defaults to 1h cache TTL when the request omits cacheRetention, with safe fallback", async () => {
@@ -1163,6 +1173,12 @@ describe("anthropic stream envelope handling", () => {
 				model,
 				{ ...model, compat: { supportsLongCacheRetention: false } },
 				{ ...model, baseUrl: "https://proxy.example.com/anthropic" },
+				{
+					...model,
+					id: "custom-compatible-model",
+					name: "Custom compatible model",
+					baseUrl: "https://proxy.example.com/anthropic",
+				},
 			]) {
 				// No cacheRetention passed: the provider default should drive the TTL.
 				const stream = streamAnthropic(testModel, context, { apiKey: "sk-ant-test" });
@@ -1185,7 +1201,10 @@ describe("anthropic stream envelope handling", () => {
 		expect(cacheControls[0]).toEqual({ type: "ephemeral", ttl: "1h" });
 		// Models without long-cache support fall back to the default ~5m breakpoint.
 		expect(cacheControls[1]).toEqual({ type: "ephemeral" });
-		// Unknown compatible endpoints do not receive generated cache controls.
-		expect(cacheControls[2]).toBeUndefined();
+		// Claude-family models through Anthropic-compatible gateways get automatic
+		// caching; without explicit long-retention support the default ~5m breakpoint applies.
+		expect(cacheControls[2]).toEqual({ type: "ephemeral" });
+		// Non-Claude models on unknown compatible endpoints receive no generated caching.
+		expect(cacheControls[3]).toBeUndefined();
 	});
 });