fix(ai): step cache breakpoints down instead of disabling caching outright

[probepark]

What

Follow-up to #3943 (merged as a884843b3). That fix stops the cache-breakpoint-overflow 400 from killing the turn, but it recovers with a kill switch: one rejection and generated caching goes off for the whole provider session. This makes the recovery graded instead.

Step	What we emit	When
2	prefix anchor + current-turn refresh point	default
1	prefix anchor only	after first overflow rejection
0	nothing	after a second rejection

Why

The rejection says "too many", not "none allowed". Turning caching off entirely on the first 400 discards more than the endpoint actually asked for.

applyExplicitPromptCaching emits two markers: an anchor on the last assistant turn, which covers the reusable conversation prefix, and a refresh point on the current turn. A gateway that leaves one free slot can accept the anchor — the marker that carries essentially all of the caching value — but under the merged behavior it gets nothing and pays full input cost on every subsequent turn.

That is the same silent-cost-regression failure mode the review of the closed #3935 flagged when rejecting a static none default. #3943 avoided it in the default path and then reintroduced it in the recovery path. This closes that gap: caching is abandoned only when the endpoint proves it has no room at all.

from/to are now logged on the step-down so the degradation is visible rather than silent.

Edge case

The prefix anchor consumes budget only when a marker is actually placed. On a first turn there is no assistant history, so a reduced budget still emits its single current-turn marker instead of silently emitting nothing.

Testing

GJC_NO_STRICT=0 bun test packages/ai/test/anthropic-cache.test.ts \
  packages/ai/test/anthropic-stream-envelope.test.ts   # 47 pass
GJC_NO_STRICT=0 bun test packages/ai/test              # 2207 pass, 1 fail (pre-existing)
bun --cwd=packages/ai run check                        # pass

Four wire-level regressions:

• graded step-down — gateway with one free slot: 2 markers rejected → retry with 1 → accepted; session state records budget 1; next turn starts at 1 without re-triggering the 400
• full give-up — gateway with zero free slots: rejects twice, 3 attempts total, ends at 0 markers and budget 0
• statusless proxy SSE form recovers identically
• unrelated 400 leaves the budget untouched at 2 and surfaces immediately

All four fail on the parent commit (a884843b3) and pass here — verified by stashing only anthropic.ts.

artifacts/issue-3670-anthropic-cache-eval.json regenerated via WRITE_ISSUE_3670_EVAL=1; it pins the provider source blob oid/sha256 by design, so it is re-derived on this base rather than carried forward.

One pre-existing failure is unrelated and reproduces on unmodified dev: no-strict-env.test.ts > keeps strict mode on when neither name is set spawns a probe without clearing an inherited PI_NO_STRICT=1 from the shell environment.

Not tested: live traffic against the reporting gateway.

GJC verdict

gajae.pr-review-verdict.v1 needs-human sha256:b10c5c219 reviewer:human evidence:local-bun-test-packages-ai

No independent architect/critic review was run.

---

• Target branch dev
• bun check passes — bun --cwd=packages/ai run check passes; full bun check blocked by an unrelated pre-existing check:sdk-closure self-test failure on dev
• Tested locally
• CHANGELOG updated (if user-facing)
• Verdict matches exact PR head, not earlier commit

[Yeachan-Heo]

Closing during the emergency maintenance freeze. This PR is not in the retained critical or maintainer-owned set. Do not open a replacement PR unless a maintainer explicitly directs it.

—
[repo owner's gaebal-gajae (clawdbot) 🦞]

[Yeachan-Heo]

Terminal review — fix(ai): step cache breakpoints down instead of disabling caching outright

Reviewer: maintainer red-team (emergency AI-subsystem lane)
Head reviewed: 9f8cff9fabc99dce4c05e0e3b2569b0482d02298
Base declared: dev @ 3765d8422
Verdict: NOT_MERGE_READY — rebase + re-run required (hold, do not merge at this head)

---

1. Diff vs. merged #3943 and current dev

#3943 (merged as a884843b, now in dev) introduced a binary kill-switch: on a A maximum of 4 blocks with cache_control may be provided rejection it set generatedCachingDisabled = true and retried once with all generated caching off, sticky for the provider session.

This PR replaces that with a graded monotonic stepdown:

type GeneratedCacheBudget = 2 | 1 | 0;

• budget 2 → normal explicit mode (conversation-prefix anchor + current-turn refresh point).
• on overflow rejection → step to 1 (keep the higher-value prefix anchor, drop the current-turn marker).
• on a second overflow rejection → step to 0 (== #3943's behavior, mode: "none").
• session-sticky in both directions; close() resets to 2.

The state shape change (generatedCachingDisabled: boolean → generatedCacheBudget: 2|1|0) is threaded through every callsite: getCacheControl, buildParams, applyPromptCaching, applyExplicitPromptCaching, the stream retry block. No stale references to the old field remain (grepped the whole tree). Rename is clean.

Red-team — is the stepdown safe?

• Termination: each rejection strictly lowers budget (2→1→0); the budget === 0 arm in getCacheControl returns mode: "none", and applyExplicitPromptCaching early-returns on budget === 0. At most two self-induced retries per turn, then the classifier stops claiming. No infinite loop, bounded exactly like #3943 plus one. ✅
• Value ordering is correct: the prefix anchor (last completed assistant turn) is spent first, the cheap current-turn refresh second. An endpoint that frees exactly one slot keeps caching the reusable prefix — strictly better than #3943's all-or-nothing. ✅
• First-turn edge case is handled: on a turn with no prior assistant message, the prefix-anchor loop places nothing and remaining stays at budget, so the reduced budget still reaches the current-turn marker instead of emitting zero markers. The new test gives up generated caching only after the reduced budget is also rejected plus the rewritten single-turn proxy-SSE test pin this. ✅
• Automatic mode is budget-correct: automatic emits exactly one top-level marker; any non-zero budget covers it, and budget === 0 returns early before the cache_control assignment. The added comment is accurate. ✅
• No widening of the claim: the overflow guard is unchanged (!fallbackManaged && firstTokenTime === undefined && isAnthropicCacheBreakpointOverflowError(...)), so unrelated 400s / non-400 / our own pre-flight failure still surface immediately. Only the recovery action changed. ✅
• Docs (docs/models.md) match the new semantics. ✅

This is a strict improvement over #3943 with no regression surface I could find. The behavior delta is: "endpoint with one free slot now keeps prefix caching" instead of "loses all generated caching on first rejection."

2. Adversarial test / CI assessment (run locally against the diff applied to current dev)

Check	Result
packages/ai/test/anthropic-stream-envelope.test.ts + anthropic-cache.test.ts	47 pass / 0 fail (incl. both new stepdown cases + rewritten overflow tests). Baseline dev = 46 pass; PR adds +1 net.
Broader matrix: anthropic-cache, anthropic-stream-envelope, anthropic-retry, anthropic-thinking-repair-retry, auth-gateway-anthropic-caching, auth-gateway-anthropic-to-codex-caching	69 pass / 0 fail (2 e2e correctly skipped: no E2E env).
bun --cwd=packages/ai run check:types (after regenerating the missing docs-index.generated, which is a pre-existing local-install artifact, identical failure on clean baseline dev)	pass (0 new type errors from this PR).
grep -rn generatedCachingDisabled (whole tree)	0 hits — rename complete.

Exact-head CI on GitHub (head 9f8cff9fa): all PR-relevant jobs pass — check:@gajae-code/ai, test:packages/ai/test/anthropic-stream-envelope.test.ts, native-build, root-check, cli-smoke, ts-build (stats + coding-agent), 5/6 cargo-build, all gjc-state-gates, Local public surfaces, Telegram daemon generation guard. One job still pending: cargo-build: pi-natives (Rust build of a crate this PR does not touch — expected to pass, but not green at the moment of review).

3. Why NOT_MERGE_READY at this head — concrete blockers

Blocker A — branch is CONFLICTING / mergeStateStatus=DIRTY vs current dev. This is the deciding factor; GitHub will not produce a clean merge of the exact head you asked me to review.

Root cause: dev advanced after the force-push at 12:30:39Z with the 0.12.15 release back-merge (473eab944 merge: bring main's corrected 0.12.15 release back into dev), which re-landed #3943 (0c836b216) and relocated its changelog entry. A real test-merge (git merge 9f8cff9fa into latest dev) conflicts in exactly two files — both bookkeeping, neither the actual code:

1. packages/ai/CHANGELOG.md — the PR rewords the cache-breakpoint entry, but that entry now lives in the released ## [0.12.15] - 2026-08-06 section (the [Unreleased] slot the PR targets is empty on current dev). Resolving this is not a blind "take theirs": per repo policy (AGENTS.md: "add entries under ## [Unreleased], never edit released sections") the contributor must rebase and place the new stepdown wording under [Unreleased], leaving the released 0.12.15 entry describing #3943's behavior untouched. This needs the contributor, not a merge-bot.
2. artifacts/issue-3670-anthropic-cache-eval.json — providerSourceBlobOid/providerSourceSha256 pins diverged (dev = e4dcc79e1a…, PR = 9e400ada3f…). This is a generated eval hash; it must be regenerated against the merged anthropic.ts via WRITE_ISSUE_3670_EVAL=1 bun test packages/ai/test/anthropic-cache-eval.integration.test.ts. Not mechanically mergeable.

The substantive files — packages/ai/src/providers/anthropic.ts and packages/ai/test/anthropic-stream-envelope.test.ts — merge cleanly. There is no semantic conflict.

Blocker B — exact-head CI not fully green. cargo-build: pi-natives is still pending at review time. It will almost certainly pass (unrelated crate), but the condition "exact head CI green" is not literally met yet.

4. Overlap check

No functional overlap blocker. Searched all open PRs targeting dev: only #3927 (ClinePass / Command Code GOAT, feat(providers)) shares a file (docs/models.md), and it does not touch anthropic.ts, the cache code, the test, or the cache-eval artifact. Its docs/models.md edit is in a different section. No conflict path overlaps #3947's code.

5. Required remediation (hold for contributor)

1. Rebase onto current dev (473eab944+) to clear the race.
2. packages/ai/CHANGELOG.md: add the stepdown wording as a new entry under ## [Unreleased]; do not edit the released ## [0.12.15] block (that entry correctly describes the shipped #3943 behavior).
3. Regenerate artifacts/issue-3670-anthropic-cache-eval.json with WRITE_ISSUE_3670_EVAL=1 bun test packages/ai/test/anthropic-cache-eval.integration.test.ts against the rebased anthropic.ts.
4. Push and let exact-head CI go green (it already passes on every job that touches this PR's files).

Once rebased conflict-free with green exact-head CI, this is MERGE_READY — the code is verified correct and is a strict improvement over merged #3943. Not merging at the current head because the two conditions above (conflict-free exact head, fully green exact-head CI) are not satisfied.

---

Signed: maintainer red-team, emergency AI-subsystem lane. Rebased re-review offered on next force-push.

[Yeachan-Heo]

Post-rebase re-review — graded cache-breakpoint stepdown

Reviewer: maintainer red-team (emergency AI-subsystem lane)
Rebased head: bceb8091e (was 9f8cff9fa)
Base: dev @ 473eab944
Verdict: MERGE_READY

This re-review follows the owner-directed rebase of this PR onto current dev. The original technical findings (adversarial code review of the generatedCacheBudget: 2|1|0 stepdown vs. merged #3943's binary kill-switch) are unchanged and still hold — see the prior review comment for the full analysis. This comment records only the post-rebase state.

Rebase resolution (owned conflicts)

The rebase from the stale base 3765d8422 onto current dev@473eab944 resolved exactly the two owned bookkeeping conflicts identified in the prior review — no code conflict:

1. packages/ai/CHANGELOG.md — the stepdown wording now lives under ## [Unreleased] (its correct home), and the released ## [0.12.15] section is left untouched describing the shipped #3943 behavior, per repo policy ("add entries under ## [Unreleased], never edit released sections").
2. artifacts/issue-3670-anthropic-cache-eval.json — regenerated against the rebased anthropic.ts via WRITE_ISSUE_3670_EVAL=1 bun test packages/ai/test/anthropic-cache-eval.integration.test.ts. Verified: providerSourceBlobOid = git rev-parse HEAD:packages/ai/src/providers/anthropic.ts = git hash-object anthropic.ts = 9e400ada3f74e4ff88652fe069aa2bd5766f9807. Consistent.

packages/ai/src/providers/anthropic.ts and the test merged cleanly (no conflict). Authorship preserved (probe <re2rar@gmail.com>). Exactly one commit on dev. git diff origin/dev..head = 5 files, +153/-44.

Local verification on rebased head

Check	Result
Test merge into dev	clean, zero conflicts
anthropic-stream-envelope + anthropic-cache + anthropic-retry + anthropic-thinking-repair-retry + anthropic-cache-eval.integration	71 pass / 0 fail
@gajae-code/ai typecheck	clean

Exact-head CI (new head bceb8091e)

Fully green: 19× success, 6× skipped (platform-specific), 0× in progress, 0× failure. Includes check:@gajae-code/ai, test:packages/ai/test/anthropic-stream-envelope.test.ts, native-build, all 6 cargo-build, both ts-build, all gjc-state-gates, root-check, cli-smoke.

Conditions for merge (owner directive)

• Terminal verdict MERGE_READY
• Exact-head CI green
• No overlap blocker (no open PR touches the cache-breakpoint code; #3927 shares only an unrelated docs/models.md section)

All three conditions satisfied.

---

Signed: maintainer red-team, emergency AI-subsystem lane. Rebased re-review complete.

[probepark]

Remediation complete — all four items addressed

Head is now bceb8091e4de09d8edf298c8ae7d2d726580f1d1, rebased onto dev @ 473eab944.

1. Rebase onto current dev ✅

git merge-base HEAD origin/dev == git rev-parse origin/dev == 473eab944. A real test-merge produces 0 conflict markers, and GitHub now reports mergeable: MERGEABLE (was CONFLICTING).

2. CHANGELOG placement ✅

The stepdown wording is a new entry under ## [Unreleased] → ### Fixed, marked (#3934, supersedes #3943). The released ## [0.12.15] block is byte-untouched and still describes the shipped #3943 kill-switch behavior — per AGENTS.md ("never edit released sections"). Verified by reading both blocks after the rebase.

3. Eval artifact regenerated ✅

Regenerated on the rebased tree, not carried forward:

WRITE_ISSUE_3670_EVAL=1 bun test packages/ai/test/anthropic-cache-eval.integration.test.ts

Note the env var is WRITE_ISSUE_3670_EVAL (renamed in #3937); the older WRITE_ARCHITECTURE_2383_EVAL spelling silently no-ops. The pin now matches the merged source:

artifact providerSourceBlobOid: 9e400ada3f74e4ff88652fe069aa2bd5766f9807
git rev-parse HEAD:packages/ai/src/providers/anthropic.ts
                 → 9e400ada3f74e4ff88652fe069aa2bd5766f9807

4. Exact-head CI ✅

23 SUCCESS / 6 SKIPPED / 0 failing / 0 pending. cargo-build: pi-natives — the job that was pending at review time — is green.

---

Local re-verification on the rebased head

Check	Result
anthropic-cache + anthropic-stream-envelope + anthropic-cache-eval.integration	49 pass / 0 fail
bun --cwd=packages/ai run check	pass
packages/ai full suite	2207 pass / 337 skip / 1 fail (pre-existing)

The one failure is no-strict-env.test.ts > keeps strict mode on when neither name is set. It reproduces identically on unmodified origin/dev (verified via detached checkout in the same shell): the test spawns a probe without clearing an inherited PI_NO_STRICT=1 from the environment. Unrelated to this PR.

Thanks for the precise remediation list — especially catching that the changelog entry needed to move rather than be force-resolved. Both conditions you named (conflict-free exact head, fully green exact-head CI) are now satisfied.

gajae.pr-review-verdict.v1 needs-human sha256:bceb8091e4de09d8edf298c8ae7d2d726580f1d1 reviewer:human evidence:local-bun-test-packages-ai + exact-head-CI-23-success

Re-review welcome.