Skip to content

fix(agent): serialize detached managed snapshots - #4580

Merged
Yeachan-Heo merged 2 commits into
devfrom
fix/issue-4578-managed-snapshot-serialization
Aug 15, 2026
Merged

fix(agent): serialize detached managed snapshots#4580
Yeachan-Heo merged 2 commits into
devfrom
fix/issue-4578-managed-snapshot-serialization

Conversation

@Yeachan-Heo

@Yeachan-Heo Yeachan-Heo commented Aug 15, 2026

Copy link
Copy Markdown
Owner

Summary

  • validate and byte-measure the detached managed snapshot rather than trusting the live payload's JSON serialization
  • sanitize the detached clone when structuredClone removes a payload class toJSON() and exposes JSON-hostile bigint state
  • surface residual local snapshot diagnostics once without same-model retry amplification

Reproducer

A provider payload class with prototype toJSON() and an own bigint field serialized live, but structuredClone erased the serializer while retaining the bigint. The previously accepted callback and terminal snapshots then failed JSON.stringify at providerPayload.envelope.sequence.

Verification

  • bun test packages/agent/test/managed-attempt-transaction.test.ts packages/coding-agent/test/agent-session-fallback-attempt-transaction.test.ts
  • bun test packages/ai/test/model-fallback-transport-facts.test.ts packages/ai/test/google-gemini-cli-alignment.test.ts packages/ai/test/openai-codex-stream.test.ts
  • bun --cwd=packages/agent run check
  • bun --cwd=packages/coding-agent run check
  • bun run build
  • affected PR planner and all selected tasks

Closes #4578

gajae.pr-review-verdict.v1 merge-approved sha256:b91f17b7e83c76c77079aa15f76de1796831a53bb6df621f73e0353f3d6fb856 reviewer:human reviewer-id:probepark evidence:#4580 (review)

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@Yeachan-Heo
Yeachan-Heo requested a review from probepark August 15, 2026 08:30
@Yeachan-Heo
Yeachan-Heo force-pushed the fix/issue-4578-managed-snapshot-serialization branch from faaa44a to a13677f Compare August 15, 2026 08:38
@Yeachan-Heo

Copy link
Copy Markdown
Owner Author

Signed supersession evidence for issue #4578

Signed-by: Yeachan-Heo
Reviewed-head: a13677f

@Yeachan-Heo

Copy link
Copy Markdown
Owner Author

Signed supersession evidence for canonical issue #4578 recovery

  • Supersedes canonical head a13677f761485c63d7188f7520e70d9d0cdecb73 / digest 84aa792794969dedd9ed2e86ad55102da98ee04541576541ead2464c75a558fe.
  • Integrated ONLY duplicate PR fix(agent): stop managed snapshot failure retry looping #4581 docs commit 04be9224886a8cdcdba123cd8643cc1354e8a6f4; the six implementation files are byte-identical to prior canonical head and duplicate implementation.
  • Exact base: 2e3ccb5895568ffec709b3adba25bc919d6d248b.
  • New seven-file exact head: a03dde8d675de017781a547c876c26a022a1d105.
  • Canonical binary diff digest: 12c4d839cb47894f7ba00361f67279c4e868ef67c8feeb4ed9e6a5f7dc5d2407.
  • Docs now match code/tests: local_snapshot_failure and local_buffer_overflow surface immediately, never retry, never charge/advance provider fallback, and never rotate credentials.
  • Fresh verification: reproducer 1/1; managed/fallback 70/70; agent and coding-agent checks pass; docs-index 5/5 after canonical local regeneration; seven-file affected planner selected suites/builds pass.
  • Generated native declaration drift restored; tracked worktree clean before push.

Signed-by: Yeachan-Heo
Reviewed-head: a03dde8

@snowykr

snowykr commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

Independent reproduction on an affected macOS host confirms the serialization primitive described by this PR.

class Envelope {
  sequence = 1n;

  toJSON() {
    return { sequence: String(this.sequence) };
  }
}

const live = { providerPayload: { envelope: new Envelope() } };
console.log(JSON.stringify(live)); // succeeds

const detached = structuredClone(live);
console.log(detached.providerPayload.envelope.constructor.name); // Object
console.log(typeof detached.providerPayload.envelope.sequence); // bigint
JSON.stringify(detached); // throws

Observed with Bun on the affected macOS host:

live {"providerPayload":{"envelope":{"sequence":"1"}}}
cloneClass Object sequenceType bigint
reproduced TypeError JSON.stringify cannot serialize BigInt.

The affected session repeatedly terminated with errorKind: "local_snapshot_failure" immediately after managed assistant events. The raw rejected provider event is not persisted after the managed transaction is discarded, so the historical payload cannot be proven byte-for-byte identical; however, the deterministic failure mechanism and rejecting snapshot boundary match this PR's JsonSafeBigIntEnvelope regression exactly.

This also supports removing the bounded same-model retry for this error: replaying an identical serializable-live/unserializable-detached payload deterministically reproduces the same local defect rather than recovering a provider failure.

@probepark probepark left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve at a03dde8d6 - and this correctly reverses something I approved

fix(agent): serialize detached managed snapshots, head commit "docs(retry): local snapshot
failures surface immediately, no retry". 7 files, +174, mergeable_state: clean.

It removes the retry branch I approved in #4550

-		if (localSnapshot) {
-			// The provisional charge was already discarded at classification time.
-			outcome = attemptsUsed <= retrySettings.maxRetries ? "retry" : "exhausted";
-		} else {
-			outcome = managedFallback ? controller.onAttemptFailure(...) : ...;
-		}
+		outcome = managedFallback ? controller.onAttemptFailure(...) : ...;
-			if (localSnapshot) {
-				// Bounded local retries exhausted: surface the original local
-				// diagnostic without any provider-fallback attribution.
-				return managedOutcome ? { type: "terminal", terminal: { stopReason: "error", messages: [message] } } : false;
-			}

#4550 gave local snapshot failures their own bounded retry path, and I approved it. The reasoning
there - a local staging limit is not a provider failure, so do not attribute it to provider
fallback - was right about attribution and wrong about retry, and I did not question the second
half.

The docs change states why plainly, and it is correct:

-Unlike snapshot failures, re-streaming the same request reproduces the same oversized response, so it is never retried
+Like snapshot failures, re-streaming the same request reproduces the same oversized response, so it is never retried

Snapshot and buffer-overflow faults are deterministic in the payload: the same response re-staged
hits the same shape rejection or the same size cap. Retrying is pure latency with no path to
success, and it delays the diagnostic the operator actually needs. Buffer overflow was already
treated this way; this makes snapshot failures consistent with it rather than leaving two policies
for one class of fault.

The retry.enabled interaction is also cleaned up:

-This opt-out also covers local snapshot failures on managed chains
+Managed provider-fallback failures keep their own chain policy; local snapshot and buffer-overflow failures surface immediately regardless of this setting

Better: whether a deterministic local fault is surfaced should not depend on a provider-retry
toggle.

Differential

# base 2e3ccb589, with this head's test files applied
(fail) ... > surfaces a typed local snapshot failure without a same-model retry
(fail) ... > surfaces a prefix-classified snapshot failure once on a single-model session
(fail) ... > surfaces one local diagnostic without deterministic snapshot-failure retries

# head a03dde8d6
 70 pass  0 fail
$ bun --cwd=packages/agent run check   -> exit 0

Three failures against three behaviours, and the names encode the policy rather than describing
mechanics - "once", "without a same-model retry", "without deterministic snapshot-failure retries".
Those are the assertions that stop the retry branch from being reintroduced.

Verdict

merge-approved.

Noting for my own record: I approved #4550 on the strength of its safety guard (do not retry a
failure carrying visible content) without asking whether retrying a deterministic failure could ever
succeed. The guard was necessary but the premise underneath it was not examined. Worth remembering
when a change adds a retry - the first question is whether the failure can plausibly resolve on
repeat, not whether the retry is bounded safely.

Reviewed by @probepark - method: source diff against the base to identify the removed retry branch, docs rationale read for the determinism argument, fresh-worktree run and agent typecheck at the exact head, separate clean base worktree with the head's test files to prove the differential.

@Yeachan-Heo
Yeachan-Heo force-pushed the fix/issue-4578-managed-snapshot-serialization branch from a03dde8 to 08fcf92 Compare August 15, 2026 09:35
@Yeachan-Heo

Copy link
Copy Markdown
Owner Author

Signed freshness supersession for issue #4578

Signed-by: Yeachan-Heo
Reviewed-head: 08fcf92

@Yeachan-Heo
Yeachan-Heo requested a review from probepark August 15, 2026 09:36
@Yeachan-Heo
Yeachan-Heo force-pushed the fix/issue-4578-managed-snapshot-serialization branch from 08fcf92 to d6540a7 Compare August 15, 2026 10:00
@Yeachan-Heo

Copy link
Copy Markdown
Owner Author

Signed freshness supersession for issue #4578

Signed-by: Yeachan-Heo
Reviewed-head: d6540a7

Yeachan Heo and others added 2 commits August 15, 2026 10:13
A payload class can serialize live through prototype toJSON while structuredClone removes that serializer and retains bigint state. Validate and measure the detached value, sanitize that exact clone when needed, and surface residual local failures once instead of replaying the same deterministic defect.

Lore-id: 4578d5a1
Constraint: preserve managed transaction isolation, provider classification, hostile payload safety, and staged byte caps
Rejected: bounded same-model retries | deterministically amplifies the same local producer defect
Confidence: high
Scope-risk: medium
Reversibility: clean
Tested: managed transaction and session cohorts; issue #2408 transport cohort; agent and coding-agent checks; workspace build; affected CI planner
The #4578 fix removed the bounded same-model retry for
local_snapshot_failure because the retained producer shape is
deterministic and retries only replayed the same defect three times.
Sync the retry-policy doc with the shipped behavior.

Lore-id: 61d94fea
Confidence: high
Scope-risk: narrow
Reversibility: clean
Tested: doc-only; behavior covered by managed-attempt and session fallback suites
@Yeachan-Heo
Yeachan-Heo force-pushed the fix/issue-4578-managed-snapshot-serialization branch from d6540a7 to 184b85a Compare August 15, 2026 10:17
@Yeachan-Heo

Copy link
Copy Markdown
Owner Author

Signed freshness supersession for issue #4578 after warned dependency merge

Signed-by: Yeachan-Heo
Reviewed-head: 184b85a

@Yeachan-Heo
Yeachan-Heo merged commit 45885ea into dev Aug 15, 2026
38 of 61 checks passed
@Yeachan-Heo

Copy link
Copy Markdown
Owner Author

Signed exact merge evidence


[repo owner's gaebal-gajae (clawdbot) 🦞]

@Yeachan-Heo
Yeachan-Heo deleted the fix/issue-4578-managed-snapshot-serialization branch August 15, 2026 10:39
Yeachan-Heo pushed a commit that referenced this pull request Aug 15, 2026
PR #4580 made local snapshot and buffer failures terminal before retry-attempt accounting. Rebasing the timeout-ceiling series retained the obsolete localSnapshot condition from the older control flow, even though that branch is now unreachable.

Match current dev's attempt accounting while retaining the provider retry-ceiling checks added by #4557.

Lore-id: issue-4464-snapshot-rebase-reconciliation
Constraint: local snapshot failures remain terminal and never consume provider fallback attempts
Rejected: retain the obsolete localSnapshot ternary | diverges from current dev after the early terminal return
Confidence: high
Scope-risk: narrow
Reversibility: trivial
Tested: managed attempt transaction and fallback transaction suites 70/70
Tested: agent-session resilient retry and fallback suites 95/95
Yeachan-Heo pushed a commit that referenced this pull request Aug 15, 2026
…ad leak

The #4580 circuit breaker surfaced local snapshot failures once, but the
producers that trip the circuit were still live. Two root causes are fixed
so the circuit does not fire on benign payloads at all:

packages/agent: a payload class carrying assistant/message-event fields on
prototype getters clones into an empty record (structuredClone copies only
own enumerable properties), so the live role/type checks passed while the
detached snapshot lost the identity and deterministically failed as
shell.role / event.unknownType. The shell and event snapshots now repair
such roots (and readable proxies) through the existing guarded-read path,
and the run-loop message_update replay normalizes through the managed event
snapshot instead of a naive spread that dropped prototype-carried fields.

packages/ai: cursor native tool calls attached raw protobuf-es payloads
(bigint fields, $typeName markers, byte arrays) as toolCall arguments,
defeating JSON.stringify in snapshot staging, transcript persistence, and
replay. Arguments are now converted to plain JSON-safe data at the provider
boundary.

Lore-id: 61d94fea
Constraint: hostile shapes (throwing get traps, sentinel-degraded content, non-string event types) keep named fail-fast diagnostics with no retry authority
Constraint: repair reads stay guarded (managedProperty) so a hostile trap can only degrade a field to undefined
Rejected: widening the sanitizer to accept unserializable staged values | hides producer defects behind lossy placeholders
Rejected: repairing hostile get-trap proxies | unreadable roots must not gain retry authority
Confidence: high
Scope-risk: medium
Reversibility: clean
Tested: payload-class end-to-end managed run; descriptor-trap proxy repair; cursor protobuf argument conversion; full agent suite (811 pass)
Not-tested: live Cursor provider session
Yeachan-Heo pushed a commit that referenced this pull request Aug 15, 2026
PR #4580 made local snapshot and buffer failures terminal before retry-attempt accounting. Rebasing the timeout-ceiling series retained the obsolete localSnapshot condition from the older control flow, even though that branch is now unreachable.

Match current dev's attempt accounting while retaining the provider retry-ceiling checks added by #4557.

Lore-id: issue-4464-snapshot-rebase-reconciliation
Constraint: local snapshot failures remain terminal and never consume provider fallback attempts
Rejected: retain the obsolete localSnapshot ternary | diverges from current dev after the early terminal return
Confidence: high
Scope-risk: narrow
Reversibility: trivial
Tested: managed attempt transaction and fallback transaction suites 70/70
Tested: agent-session resilient retry and fallback suites 95/95
Yeachan-Heo pushed a commit that referenced this pull request Aug 15, 2026
…ad leak

The #4580 circuit breaker surfaced local snapshot failures once, but the
producers that trip the circuit were still live. Two root causes are fixed
so the circuit does not fire on benign payloads at all:

packages/agent: a payload class carrying assistant/message-event fields on
prototype getters clones into an empty record (structuredClone copies only
own enumerable properties), so the live role/type checks passed while the
detached snapshot lost the identity and deterministically failed as
shell.role / event.unknownType. The shell and event snapshots now repair
such roots (and readable proxies) through the existing guarded-read path,
and the run-loop message_update replay normalizes through the managed event
snapshot instead of a naive spread that dropped prototype-carried fields.

packages/ai: cursor native tool calls attached raw protobuf-es payloads
(bigint fields, $typeName markers, byte arrays) as toolCall arguments,
defeating JSON.stringify in snapshot staging, transcript persistence, and
replay. Arguments are now converted to plain JSON-safe data at the provider
boundary.

Lore-id: 61d94fea
Constraint: hostile shapes (throwing get traps, sentinel-degraded content, non-string event types) keep named fail-fast diagnostics with no retry authority
Constraint: repair reads stay guarded (managedProperty) so a hostile trap can only degrade a field to undefined
Rejected: widening the sanitizer to accept unserializable staged values | hides producer defects behind lossy placeholders
Rejected: repairing hostile get-trap proxies | unreadable roots must not gain retry authority
Confidence: high
Scope-risk: medium
Reversibility: clean
Tested: payload-class end-to-end managed run; descriptor-trap proxy repair; cursor protobuf argument conversion; full agent suite (811 pass)
Not-tested: live Cursor provider session
Yeachan-Heo pushed a commit that referenced this pull request Aug 15, 2026
PR #4580 made local snapshot and buffer failures terminal before retry-attempt accounting. Rebasing the timeout-ceiling series retained the obsolete localSnapshot condition from the older control flow, even though that branch is now unreachable.

Match current dev's attempt accounting while retaining the provider retry-ceiling checks added by #4557.

Lore-id: issue-4464-snapshot-rebase-reconciliation
Constraint: local snapshot failures remain terminal and never consume provider fallback attempts
Rejected: retain the obsolete localSnapshot ternary | diverges from current dev after the early terminal return
Confidence: high
Scope-risk: narrow
Reversibility: trivial
Tested: managed attempt transaction and fallback transaction suites 70/70
Tested: agent-session resilient retry and fallback suites 95/95
Yeachan-Heo pushed a commit that referenced this pull request Aug 15, 2026
PR #4580 made local snapshot and buffer failures terminal before retry-attempt accounting. Rebasing the timeout-ceiling series retained the obsolete localSnapshot condition from the older control flow, even though that branch is now unreachable.

Match current dev's attempt accounting while retaining the provider retry-ceiling checks added by #4557.

Lore-id: issue-4464-snapshot-rebase-reconciliation
Constraint: local snapshot failures remain terminal and never consume provider fallback attempts
Rejected: retain the obsolete localSnapshot ternary | diverges from current dev after the early terminal return
Confidence: high
Scope-risk: narrow
Reversibility: trivial
Tested: managed attempt transaction and fallback transaction suites 70/70
Tested: agent-session resilient retry and fallback suites 95/95
Yeachan-Heo pushed a commit that referenced this pull request Aug 15, 2026
PR #4580 made local snapshot and buffer failures terminal before retry-attempt accounting. Rebasing the timeout-ceiling series retained the obsolete localSnapshot condition from the older control flow, even though that branch is now unreachable.

Match current dev's attempt accounting while retaining the provider retry-ceiling checks added by #4557.

Lore-id: issue-4464-snapshot-rebase-reconciliation
Constraint: local snapshot failures remain terminal and never consume provider fallback attempts
Rejected: retain the obsolete localSnapshot ternary | diverges from current dev after the early terminal return
Confidence: high
Scope-risk: narrow
Reversibility: trivial
Tested: managed attempt transaction and fallback transaction suites 70/70
Tested: agent-session resilient retry and fallback suites 95/95
Yeachan-Heo added a commit that referenced this pull request Aug 15, 2026
* fix(anthropic): bound first-event timeout retries

Large Anthropic requests could spend a full first-event window and then be uploaded again even when a proxy was already load-shedding upstream. Carry safe timeout facts across retry boundaries, give custom endpoints a bounded observation grace, and cap replay attempts by serialized payload size.

Lore-id: issue-4464

Constraint: never expose request body, URL credentials, or query tokens in timeout diagnostics

Constraint: multi-megabyte full-window timeouts must not trigger another upload

Rejected: increase the global timeout | leaves dead connections hanging and does not bound replay amplification

Confidence: high

Scope-risk: medium

Reversibility: straightforward

Tested: deterministic Anthropic timeout, late 529, payload-size, abort, redaction, retry-accounting, and package type checks

Not-tested: live third-party proxy behavior

Directive: preserve @probepark attribution for issue #4464

* fix(anthropic): narrow proxy timeout grace

Apply the bounded observation grace only to large requests on custom endpoints, so small proxied requests keep their configured deadline. Enforce provider attempt ceilings before managed fallback state mutates and report the same ceiling in retry events.

Lore-id: issue-4464-fix-forward

Constraint: large full-window timeouts must not upload again

Constraint: small custom requests must keep their configured first-event deadline

Confidence: high

Scope-risk: medium

Reversibility: straightforward

Tested: deterministic late 529, grace expiry, redaction, abort, payload threshold, retry accounting, focused regressions, package checks

Directive: preserve @probepark attribution for issue #4464

* fix(anthropic): preserve timeout disable and global ceiling

Keep explicit zero first-event timeouts disabled before custom-endpoint grace is considered, and leave typed timeout replays to the session-level attempt ceiling so provider and session retries cannot multiply each other.

Lore-id: issue-4464-review-fixes

Constraint: total small-timeout uploads must not exceed two across provider and session layers

Constraint: streamFirstEventTimeoutMs=0 must remain disabled

Confidence: high

Scope-risk: medium

Tested: 15 Anthropic timeout cases, 171 focused regressions, AI and coding-agent package checks

Directive: preserve @probepark attribution for issue #4464

* fix(session): retain first-event retry ceilings

Carry the provider ceiling across later transient failures in the same retry run, stop same-model amplification at the ceiling, and still advance managed fallback chains to a different model. Preserve total elapsed diagnostics across mixed failure classes.

Lore-id: issue-4464-cross-event-ceiling

Constraint: timeout-derived ceilings survive later 529 and transient failures

Constraint: managed fallback may advance but must not retry the exhausted selector

Confidence: high

Scope-risk: medium

Tested: mixed timeout-to-529 ceiling, managed model advance, 90 session tests, 26 AI timeout tests, affected package checks

Directive: preserve @probepark attribution for issue #4464

* fix(anthropic): classify canonical endpoints strictly

Treat non-default ports, URL credentials, query parameters, fragments, and non-root paths as custom endpoints even when the hostname is api.anthropic.com, so proxy grace and redaction policy cannot be bypassed by a canonical-looking host.

Lore-id: issue-4464-endpoint-class

Constraint: canonical means the credential-free default Anthropic origin only

Confidence: high

Scope-risk: narrow

Reversibility: straightforward

Tested: credentialed api.anthropic.com:8443 late-529 stub, 26 AI tests, 90 session tests, affected package checks

Directive: preserve @probepark attribution for issue #4464

* fix(anthropic): bind grace failures to retry ceilings

Attach the large-request ceiling to late 529s observed during grace, classify injected clients by their actual base URL, and clear the session ceiling immediately after a successful retry before tool-result continuations.

Lore-id: issue-4464-final-review

Constraint: late grace failures must not re-upload oversized bodies

Constraint: successful retry runs must not constrain independent continuations

Confidence: high

Scope-risk: medium

Tested: 26 AI timeout tests, 91 session retry/fallback tests, affected package checks

Directive: preserve @probepark attribution for issue #4464

* fix(stream): preserve absolute first-event ceilings

Retain statusless grace retry facts as transport evidence and measure the first semantic event against one absolute deadline, so socket failures keep their upload ceiling and preamble pings cannot rearm the watchdog indefinitely.

Lore-id: issue-4464-absolute-deadline

Constraint: non-progress events must not extend the first-event window

Constraint: statusless grace failures must retain retry ceilings

Confidence: high

Scope-risk: medium

Tested: statusless socket failure after grace, periodic pre-message_start ping, 29 AI tests, 91 session tests, affected package checks

Directive: preserve @probepark attribution for issue #4464

* fix(stream): enforce semantic first-event boundaries

Check the absolute deadline again before accepting buffered preamble items, and attach grace ceilings only while the first semantic event remains pending. This closes synchronous keepalive starvation without misclassifying post-message_start failures.

Lore-id: issue-4464-semantic-deadline

Constraint: buffered non-progress items cannot cross the absolute deadline

Constraint: grace ceilings apply only before semantic progress

Confidence: high

Scope-risk: medium

Tested: synchronous buffered ping expiry, periodic pre-message_start pings, post-message_start 529 exclusion, PI timeout zero, 55 AI tests, 91 session tests, affected package checks

Directive: preserve @probepark attribution for issue #4464

* fix(session): report grace retry ceilings

Use retained provider ceilings when rendering retry limits for late grace failures and distinguish grace-derived exhaustion from direct first-event timeout exhaustion without changing legacy timeout event counts.

Lore-id: issue-4464-grace-accounting

Constraint: retry event and terminal diagnostics must match enforced ceilings

Confidence: high

Scope-risk: narrow

Tested: late 529 grace ceiling diagnostics, 92 session tests, 55 AI tests, affected package checks

Directive: preserve @probepark attribution for issue #4464

* fix(session): prioritize upload ceilings over rotation

Compute the provider ceiling before any credential mutation and suppress both managed and non-managed same-model rotation once the bounded upload count is reached, while still surfacing exact exhaustion diagnostics.

Lore-id: issue-4464-rotation-ceiling

Constraint: a fresh credential cannot bypass an oversized-request ceiling

Confidence: high

Scope-risk: medium

Tested: two-credential late-429 grace regression, 93 session tests, 55 AI tests, affected package checks

Directive: preserve @probepark attribution for issue #4464

* fix(session): avoid post-ceiling credential mutation

Skip the managed continuation's credential failure mutation after an upload ceiling is reached, preserving affinity while advancing to another model. Cover the N>1 fallback chain with two stored credentials.

Lore-id: issue-4464-managed-affinity

Constraint: bounded upload exhaustion cannot block or rotate the credential after fallback advancement

Confidence: high

Scope-risk: narrow

Tested: managed two-credential fallback regression, 94 session tests, 55 AI tests, affected package checks

Directive: preserve @probepark attribution for issue #4464

* fix(anthropic): close provider replay bypasses

Disable Anthropic SDK request retries for large custom uploads governed by a first-event ceiling, and terminalize stamped grace failures before any strict/tool/cache/thinking corrective replay can resend the body.

Lore-id: issue-4464-provider-replay

Constraint: request SDK and corrective branches cannot bypass upload ceilings

Confidence: high

Scope-risk: medium

Tested: prestream 529 SDK retry suppression, delayed strict grammar corrective suppression, 57 AI tests, 94 session tests, affected package checks

Directive: preserve @probepark attribution for issue #4464

* fix(anthropic): validate progress envelope order

Do not clear the first-event ceiling for content or terminal events that arrive before message_start; invalid preambles now retain the large-request ceiling and cannot enter transient envelope replay.

Lore-id: issue-4464-envelope-order

Constraint: semantic progress requires valid message_start ordering

Confidence: high

Scope-risk: narrow

Tested: delayed pre-message_start content regression, 58 AI tests, 94 session tests, affected package checks

Directive: preserve @probepark attribution for issue #4464

* fix(retry): align grace timing and manual resets

Measure first-event grace from stream iteration rather than request setup, and clear provider ceilings at explicit retry boundaries and every successful assistant response so tool continuations cannot inherit stale limits.

Lore-id: issue-4464-timing-reset

Constraint: pre-stream setup time is not semantic first-event elapsed

Constraint: explicit retry starts with a fresh provider ceiling

Confidence: high

Scope-risk: narrow

Tested: 98 manual/session/fallback tests, 58 AI timeout tests, affected package checks

Directive: preserve @probepark attribution for issue #4464

* fix(anthropic): start grace clock at stream iteration

Exclude withResponse and response-hook setup from semantic first-event elapsed so immediate stream failures are not incorrectly treated as post-window grace failures.

Lore-id: issue-4464-grace-clock

Constraint: grace elapsed must match the iterator watchdog start

Confidence: high

Scope-risk: narrow

Tested: delayed response setup immediate-529 regression, 59 AI tests, 98 session/manual retry tests, affected package checks

Directive: preserve @probepark attribution for issue #4464

* test(session): clear provider ceiling on manual retry

Codex finding (P2, reviewed commit 94e0314): a failed turn carrying
retryMaxAttempts reissued through the public retry() method kept the
stale #providerRetryMaxAttempts, so a later transient failure on the
reissued turn was exhausted by the prior turn's ceiling. The fix at the
PR head clears the ceiling in retry(), but no discriminating regression
test covered it; this adds one that fails when the reset is removed.

Lore-id: issue-4464-manual-retry-ceiling
Constraint: manual retry must start a fresh attempt budget
Confidence: high
Scope-risk: narrow
Reversibility: trivial
Directive: preserve @probepark attribution for issue #4464
Tested: bun test agent-session-resilient-retry.test.ts 65 pass; fails 1 without the retry() ceiling reset
Not-tested: none

* fix(anthropic): bound outer provider retries for ceiling-bound pre-iteration failures

Codex finding 3784611300 (P1, reviewed at f236aff): a multi-megabyte
custom-endpoint request that fails before stream iteration begins (an
immediate 529 from withResponse) had the SDK's internal retries disabled
but no ceiling facts attached, because firstEventWaitStartedAt never
started. The outer provider retry loop then re-uploaded the body up to
the default streamMaxRetries budget (measured locally: attempts=4).

The catch path now stamps the one-attempt upload ceiling whenever a
ceiling-bound upload fails before iteration starts, so both retry layers
honor it. Once iteration has begun, only the grace-clock path decides,
preserving the delayed-setup contract from 25ff835.

The delayed-response-setup test used a 1ms window against a 5ms setup
sleep, so the boundary verdict depended on whether the unavoidable 2-3ms
async-throw propagation exceeded the window - CI observed both outcomes
at the same head. It is rescaled to setup 300ms / window 100ms where the
contract is decidable, verified to still fail against the pre-25ff835509
clock placement, and a complementary test proves an in-window 529 after
delayed setup keeps its ordinary provider retry budget.

Lore-id: issue-4464-outer-retry-ceiling
Constraint: ceiling must bound SDK and outer retries without suppressing in-window transients
Confidence: high
Scope-risk: narrow
Reversibility: trivial
Directive: preserve @probepark attribution for issue #4464
Tested: 6x full file 29 pass 0 fail; targeted zero/abort/large-body/retry-after suites pass; packages/ai check clean
Not-tested: live upstream 529 pacing

* fix(anthropic): normalize unknown stream rejections before stamping retry facts

Codex finding 3785717275 (P2, posted at rebased head 3520dc9): an
injected custom client (options.client is a supported surface) rejecting
withResponse() with a primitive string had the ceiling facts assigned
onto a temporary boxed value, silently discarding them. The request then
slipped past the one-attempt upload ceiling, and a string-matched
corrective branch (CPA tool-alias restore, which accepts non-Error
values via String(error)) re-uploaded the multi-megabyte body. Measured
live: attempts=2, transportFailure undefined.

The catch path now normalizes unknown rejections to a mutable Error
before any facts attach, matching the Error-instance discipline of
attachAnthropicGraceFailureFacts. Discriminating regression test added
(fails with attempts=2 when the normalization is removed).

Lore-id: issue-4464-normalize-stream-failure
Constraint: facts must survive every rejection shape from injected clients
Confidence: high
Scope-risk: narrow
Reversibility: trivial
Directive: preserve @probepark attribution for issue #4464
Tested: new test attempts=1 with stamped facts; 6x 30 pass 0 fail timeout+iterator suites; 95/95 session retry suites; both package checks clean
Not-tested: non-string non-Error rejections beyond the String() round-trip

* fix(anthropic): count consumed provider replays in the timeout ceiling

Codex finding 3786729761 (P1, posted at e443b4f at 19:51:09Z): for a
small request whose first upload received an immediate retryable 529 and
whose provider replay then hit the first-event timeout,
createAnthropicFirstEventTimeoutError derived retryMaxAttempts purely
from request size (2), even though providerRetryAttempt was already 1.
The session counts a whole provider invocation as one attempt, so the
full ceiling licensed a third upload of the same body.

The timeout error now receives providerAttemptsConsumed and reports the
remaining total ceiling (floor 1). Verified live: 529-then-timeout now
reports retryMaxAttempts 1 (was 2); discriminating regression test added
that fails with 2 when the subtraction is removed.

Lore-id: issue-4464-provider-replay-ceiling
Constraint: reported ceiling must bound total uploads across the invocation
Confidence: high
Scope-risk: narrow
Reversibility: trivial
Directive: preserve @probepark attribution for issue #4464
Tested: new test attempts=2 ceiling=1; timeout+iterator suites 31/31 x4; 95/95 session retry suites; both package checks clean after biome fix
Not-tested: managed-fallback accounting interplay beyond existing suites

* fix(anthropic): preserve transport metadata on structured non-Error rejections

Codex finding 3786938429 (P2, posted at cc110df at 20:24:07Z): the
normalizeStreamFailure helper added for primitive-string rejections also
wrapped structured rejections such as { status: 529, error: { type },
headers } from injected clients, collapsing them to Error('[object
Object]') and discarding every transport field — so status/provider-code
extraction and managed-fallback classification lost the failure class.

Structured objects are now wrapped in a mutable Error that copies every
enumerable own property (message via a cycle-safe JSON.stringify with
String fallback); primitives keep the plain String(error) wrapper.
Regression test asserts status 529 + anthropicErrorType survive with
the ceiling facts intact.

Lore-id: issue-4464-structured-rejection-metadata
Constraint: normalization must never drop transport metadata
Confidence: high
Scope-risk: narrow
Reversibility: trivial
Directive: preserve @probepark attribution for issue #4464
Tested: new test passes; timeout+iterator suites 32/32 x4; both package checks clean
Not-tested: getters-only rejection objects

* fix(anthropic): count corrective-policy replays in the timeout ceiling

Codex finding 3787055464 (P2, posted at 2446999 at 20:45:41Z): the
providerAttemptsConsumed fix read providerRetryAttempt, but the four
corrective-policy branches (strict-tool fallback, forced-tool drop,
fast-mode drop, thinking/CPA repair) reset providerRetryAttempt = 0
before continue. A small request that took one corrective replay and
whose corrected upload then timed out still reported the full
two-attempt ceiling despite two uploads already spent, licensing a
third upload of the corrected request.

A separate providerUploadCount counter now tracks total uploads across
the invocation: bumped on every provider retry and every corrective
continue, never reset, and read by the timeout facts. Regression test
covers strict-grammar corrective replay then timeout and asserts the
remaining ceiling is 1 (fails with 2 when the corrective bumps are
removed).

Lore-id: issue-4464-corrective-replay-ceiling
Constraint: ceiling must bound total uploads including corrective replays
Confidence: high
Scope-risk: narrow
Reversibility: trivial
Directive: preserve @probepark attribution for issue #4464
Tested: new test fails without the bumps; timeout+iterator suites 33/33 x4; 95/95 session retry suites; both package checks clean
Not-tested: exotic custom corrective chains beyond the four branches

* fix(anthropic): count thinking and CPA corrective uploads in the timeout ceiling

Codex finding 3787172110 (P2, posted at 84b5378 at 21:08:27Z): the
providerUploadCount counter covered strict-tool, forced-tool, fast-mode,
cache-breakpoint, and generic retries, but not the thinking-replay
repair branch or the CPA alias-repair branch — both corrective
continues that upload a corrected body without resetting the provider
retry budget. A small request whose corrected upload then timed out
still saw count 0 and reported the full two-attempt ceiling, licensing
a third upload.

Both branches now bump providerUploadCount before their replay. All
seven upload sites (initial retries x1, strict/forced/fast/cache x4,
thinking repair, CPA repair) feed the timeout ceiling.

Lore-id: issue-4464-thinking-cpa-upload-count
Constraint: every corrected-body upload must consume the ceiling
Confidence: high
Scope-risk: narrow
Reversibility: trivial
Directive: preserve @probepark attribution for issue #4464
Tested: timeout+iterator suites 33/33 x3; 95/95 session retry suites; packages/ai check clean
Not-tested: dedicated thinking/CPA corrective-then-timeout e2e beyond the shared ceiling path

* test(anthropic): pin thinking and CPA repair uploads against the timeout ceiling

The head commit of this PR marked dedicated thinking/CPA corrective-then-timeout
coverage as Not-tested. Without it, the providerUploadCount increments those
branches added had no direct guard: deleting either one would leave the shared
corrective-policy test green while the ceiling silently licensed a third upload
again — the exact issue #4464 amplification this PR exists to bound.

Both new stubs reproduce the branch-specific rejection first (invalid thinking
signature 400; CPA alias-restore 500), let the repaired body upload, then hang
that upload until the first-event timeout. They assert attempts=2 and the
remaining retryMaxAttempts=1, and both fail with retryMaxAttempts=2 on the
pre-fix parent 84b5378, proving the counter increments are load-bearing.

Lore-id: issue-4464-repair-upload-coverage
Constraint: coverage must fail if any corrective upload site stops counting
Confidence: high
Scope-risk: narrow
Reversibility: trivial
Tested: timeout suite 30/30 x3; repair/cpa/iterator cohorts 44/44; packages/ai check clean
Not-tested: live CPA proxy replay of a steered body

* test(anthropic): isolate corrective timeout replay

The timeout regression used strict-tool fallback even though GJC_NO_STRICT and PI_NO_STRICT intentionally remove strict markers before upload. Reviewers with either supported operator flag therefore saw one upload and a deterministic false failure before the retry-ceiling assertion ran.

Exercise the same resettable corrective-policy path through forced tool-choice fallback, which is independent of strict-schema configuration and still proves the second upload consumes the first-event timeout ceiling.

Lore-id: issue-4464-corrective-test-isolation
Constraint: timeout accounting coverage must pass under supported strict-schema bypass flags
Rejected: change production strict bypass semantics | the operator flags intentionally suppress strict requests
Confidence: high
Scope-risk: narrow
Reversibility: trivial
Tested: anthropic timeout suite 30/30 x5 default, 30/30 x5 GJC_NO_STRICT, 30/30 x5 PI_NO_STRICT
Tested: relevant AI suites 171/171; session retry suites 95/95; packages/ai check clean

* fix(session): preserve snapshot terminalization after rebase

PR #4580 made local snapshot and buffer failures terminal before retry-attempt accounting. Rebasing the timeout-ceiling series retained the obsolete localSnapshot condition from the older control flow, even though that branch is now unreachable.

Match current dev's attempt accounting while retaining the provider retry-ceiling checks added by #4557.

Lore-id: issue-4464-snapshot-rebase-reconciliation
Constraint: local snapshot failures remain terminal and never consume provider fallback attempts
Rejected: retain the obsolete localSnapshot ternary | diverges from current dev after the early terminal return
Confidence: high
Scope-risk: narrow
Reversibility: trivial
Tested: managed attempt transaction and fallback transaction suites 70/70
Tested: agent-session resilient retry and fallback suites 95/95

---------

Co-authored-by: Yeachan Heo <yeachan.heo@gmail.com>
Yeachan-Heo added a commit that referenced this pull request Aug 16, 2026
* fix(anthropic): bound first-event timeout retries

Large Anthropic requests could spend a full first-event window and then be uploaded again even when a proxy was already load-shedding upstream. Carry safe timeout facts across retry boundaries, give custom endpoints a bounded observation grace, and cap replay attempts by serialized payload size.

Lore-id: issue-4464

Constraint: never expose request body, URL credentials, or query tokens in timeout diagnostics

Constraint: multi-megabyte full-window timeouts must not trigger another upload

Rejected: increase the global timeout | leaves dead connections hanging and does not bound replay amplification

Confidence: high

Scope-risk: medium

Reversibility: straightforward

Tested: deterministic Anthropic timeout, late 529, payload-size, abort, redaction, retry-accounting, and package type checks

Not-tested: live third-party proxy behavior

Directive: preserve @probepark attribution for issue #4464

* fix(anthropic): narrow proxy timeout grace

Apply the bounded observation grace only to large requests on custom endpoints, so small proxied requests keep their configured deadline. Enforce provider attempt ceilings before managed fallback state mutates and report the same ceiling in retry events.

Lore-id: issue-4464-fix-forward

Constraint: large full-window timeouts must not upload again

Constraint: small custom requests must keep their configured first-event deadline

Confidence: high

Scope-risk: medium

Reversibility: straightforward

Tested: deterministic late 529, grace expiry, redaction, abort, payload threshold, retry accounting, focused regressions, package checks

Directive: preserve @probepark attribution for issue #4464

* fix(anthropic): preserve timeout disable and global ceiling

Keep explicit zero first-event timeouts disabled before custom-endpoint grace is considered, and leave typed timeout replays to the session-level attempt ceiling so provider and session retries cannot multiply each other.

Lore-id: issue-4464-review-fixes

Constraint: total small-timeout uploads must not exceed two across provider and session layers

Constraint: streamFirstEventTimeoutMs=0 must remain disabled

Confidence: high

Scope-risk: medium

Tested: 15 Anthropic timeout cases, 171 focused regressions, AI and coding-agent package checks

Directive: preserve @probepark attribution for issue #4464

* fix(session): retain first-event retry ceilings

Carry the provider ceiling across later transient failures in the same retry run, stop same-model amplification at the ceiling, and still advance managed fallback chains to a different model. Preserve total elapsed diagnostics across mixed failure classes.

Lore-id: issue-4464-cross-event-ceiling

Constraint: timeout-derived ceilings survive later 529 and transient failures

Constraint: managed fallback may advance but must not retry the exhausted selector

Confidence: high

Scope-risk: medium

Tested: mixed timeout-to-529 ceiling, managed model advance, 90 session tests, 26 AI timeout tests, affected package checks

Directive: preserve @probepark attribution for issue #4464

* fix(anthropic): classify canonical endpoints strictly

Treat non-default ports, URL credentials, query parameters, fragments, and non-root paths as custom endpoints even when the hostname is api.anthropic.com, so proxy grace and redaction policy cannot be bypassed by a canonical-looking host.

Lore-id: issue-4464-endpoint-class

Constraint: canonical means the credential-free default Anthropic origin only

Confidence: high

Scope-risk: narrow

Reversibility: straightforward

Tested: credentialed api.anthropic.com:8443 late-529 stub, 26 AI tests, 90 session tests, affected package checks

Directive: preserve @probepark attribution for issue #4464

* fix(anthropic): bind grace failures to retry ceilings

Attach the large-request ceiling to late 529s observed during grace, classify injected clients by their actual base URL, and clear the session ceiling immediately after a successful retry before tool-result continuations.

Lore-id: issue-4464-final-review

Constraint: late grace failures must not re-upload oversized bodies

Constraint: successful retry runs must not constrain independent continuations

Confidence: high

Scope-risk: medium

Tested: 26 AI timeout tests, 91 session retry/fallback tests, affected package checks

Directive: preserve @probepark attribution for issue #4464

* fix(stream): preserve absolute first-event ceilings

Retain statusless grace retry facts as transport evidence and measure the first semantic event against one absolute deadline, so socket failures keep their upload ceiling and preamble pings cannot rearm the watchdog indefinitely.

Lore-id: issue-4464-absolute-deadline

Constraint: non-progress events must not extend the first-event window

Constraint: statusless grace failures must retain retry ceilings

Confidence: high

Scope-risk: medium

Tested: statusless socket failure after grace, periodic pre-message_start ping, 29 AI tests, 91 session tests, affected package checks

Directive: preserve @probepark attribution for issue #4464

* fix(stream): enforce semantic first-event boundaries

Check the absolute deadline again before accepting buffered preamble items, and attach grace ceilings only while the first semantic event remains pending. This closes synchronous keepalive starvation without misclassifying post-message_start failures.

Lore-id: issue-4464-semantic-deadline

Constraint: buffered non-progress items cannot cross the absolute deadline

Constraint: grace ceilings apply only before semantic progress

Confidence: high

Scope-risk: medium

Tested: synchronous buffered ping expiry, periodic pre-message_start pings, post-message_start 529 exclusion, PI timeout zero, 55 AI tests, 91 session tests, affected package checks

Directive: preserve @probepark attribution for issue #4464

* fix(session): report grace retry ceilings

Use retained provider ceilings when rendering retry limits for late grace failures and distinguish grace-derived exhaustion from direct first-event timeout exhaustion without changing legacy timeout event counts.

Lore-id: issue-4464-grace-accounting

Constraint: retry event and terminal diagnostics must match enforced ceilings

Confidence: high

Scope-risk: narrow

Tested: late 529 grace ceiling diagnostics, 92 session tests, 55 AI tests, affected package checks

Directive: preserve @probepark attribution for issue #4464

* fix(session): prioritize upload ceilings over rotation

Compute the provider ceiling before any credential mutation and suppress both managed and non-managed same-model rotation once the bounded upload count is reached, while still surfacing exact exhaustion diagnostics.

Lore-id: issue-4464-rotation-ceiling

Constraint: a fresh credential cannot bypass an oversized-request ceiling

Confidence: high

Scope-risk: medium

Tested: two-credential late-429 grace regression, 93 session tests, 55 AI tests, affected package checks

Directive: preserve @probepark attribution for issue #4464

* fix(session): avoid post-ceiling credential mutation

Skip the managed continuation's credential failure mutation after an upload ceiling is reached, preserving affinity while advancing to another model. Cover the N>1 fallback chain with two stored credentials.

Lore-id: issue-4464-managed-affinity

Constraint: bounded upload exhaustion cannot block or rotate the credential after fallback advancement

Confidence: high

Scope-risk: narrow

Tested: managed two-credential fallback regression, 94 session tests, 55 AI tests, affected package checks

Directive: preserve @probepark attribution for issue #4464

* fix(anthropic): close provider replay bypasses

Disable Anthropic SDK request retries for large custom uploads governed by a first-event ceiling, and terminalize stamped grace failures before any strict/tool/cache/thinking corrective replay can resend the body.

Lore-id: issue-4464-provider-replay

Constraint: request SDK and corrective branches cannot bypass upload ceilings

Confidence: high

Scope-risk: medium

Tested: prestream 529 SDK retry suppression, delayed strict grammar corrective suppression, 57 AI tests, 94 session tests, affected package checks

Directive: preserve @probepark attribution for issue #4464

* fix(anthropic): validate progress envelope order

Do not clear the first-event ceiling for content or terminal events that arrive before message_start; invalid preambles now retain the large-request ceiling and cannot enter transient envelope replay.

Lore-id: issue-4464-envelope-order

Constraint: semantic progress requires valid message_start ordering

Confidence: high

Scope-risk: narrow

Tested: delayed pre-message_start content regression, 58 AI tests, 94 session tests, affected package checks

Directive: preserve @probepark attribution for issue #4464

* fix(retry): align grace timing and manual resets

Measure first-event grace from stream iteration rather than request setup, and clear provider ceilings at explicit retry boundaries and every successful assistant response so tool continuations cannot inherit stale limits.

Lore-id: issue-4464-timing-reset

Constraint: pre-stream setup time is not semantic first-event elapsed

Constraint: explicit retry starts with a fresh provider ceiling

Confidence: high

Scope-risk: narrow

Tested: 98 manual/session/fallback tests, 58 AI timeout tests, affected package checks

Directive: preserve @probepark attribution for issue #4464

* fix(anthropic): start grace clock at stream iteration

Exclude withResponse and response-hook setup from semantic first-event elapsed so immediate stream failures are not incorrectly treated as post-window grace failures.

Lore-id: issue-4464-grace-clock

Constraint: grace elapsed must match the iterator watchdog start

Confidence: high

Scope-risk: narrow

Tested: delayed response setup immediate-529 regression, 59 AI tests, 98 session/manual retry tests, affected package checks

Directive: preserve @probepark attribution for issue #4464

* test(session): clear provider ceiling on manual retry

Codex finding (P2, reviewed commit 94e0314): a failed turn carrying
retryMaxAttempts reissued through the public retry() method kept the
stale #providerRetryMaxAttempts, so a later transient failure on the
reissued turn was exhausted by the prior turn's ceiling. The fix at the
PR head clears the ceiling in retry(), but no discriminating regression
test covered it; this adds one that fails when the reset is removed.

Lore-id: issue-4464-manual-retry-ceiling
Constraint: manual retry must start a fresh attempt budget
Confidence: high
Scope-risk: narrow
Reversibility: trivial
Directive: preserve @probepark attribution for issue #4464
Tested: bun test agent-session-resilient-retry.test.ts 65 pass; fails 1 without the retry() ceiling reset
Not-tested: none

* fix(anthropic): bound outer provider retries for ceiling-bound pre-iteration failures

Codex finding 3784611300 (P1, reviewed at f236aff): a multi-megabyte
custom-endpoint request that fails before stream iteration begins (an
immediate 529 from withResponse) had the SDK's internal retries disabled
but no ceiling facts attached, because firstEventWaitStartedAt never
started. The outer provider retry loop then re-uploaded the body up to
the default streamMaxRetries budget (measured locally: attempts=4).

The catch path now stamps the one-attempt upload ceiling whenever a
ceiling-bound upload fails before iteration starts, so both retry layers
honor it. Once iteration has begun, only the grace-clock path decides,
preserving the delayed-setup contract from 25ff835.

The delayed-response-setup test used a 1ms window against a 5ms setup
sleep, so the boundary verdict depended on whether the unavoidable 2-3ms
async-throw propagation exceeded the window - CI observed both outcomes
at the same head. It is rescaled to setup 300ms / window 100ms where the
contract is decidable, verified to still fail against the pre-25ff835509
clock placement, and a complementary test proves an in-window 529 after
delayed setup keeps its ordinary provider retry budget.

Lore-id: issue-4464-outer-retry-ceiling
Constraint: ceiling must bound SDK and outer retries without suppressing in-window transients
Confidence: high
Scope-risk: narrow
Reversibility: trivial
Directive: preserve @probepark attribution for issue #4464
Tested: 6x full file 29 pass 0 fail; targeted zero/abort/large-body/retry-after suites pass; packages/ai check clean
Not-tested: live upstream 529 pacing

* fix(anthropic): normalize unknown stream rejections before stamping retry facts

Codex finding 3785717275 (P2, posted at rebased head 3520dc9): an
injected custom client (options.client is a supported surface) rejecting
withResponse() with a primitive string had the ceiling facts assigned
onto a temporary boxed value, silently discarding them. The request then
slipped past the one-attempt upload ceiling, and a string-matched
corrective branch (CPA tool-alias restore, which accepts non-Error
values via String(error)) re-uploaded the multi-megabyte body. Measured
live: attempts=2, transportFailure undefined.

The catch path now normalizes unknown rejections to a mutable Error
before any facts attach, matching the Error-instance discipline of
attachAnthropicGraceFailureFacts. Discriminating regression test added
(fails with attempts=2 when the normalization is removed).

Lore-id: issue-4464-normalize-stream-failure
Constraint: facts must survive every rejection shape from injected clients
Confidence: high
Scope-risk: narrow
Reversibility: trivial
Directive: preserve @probepark attribution for issue #4464
Tested: new test attempts=1 with stamped facts; 6x 30 pass 0 fail timeout+iterator suites; 95/95 session retry suites; both package checks clean
Not-tested: non-string non-Error rejections beyond the String() round-trip

* fix(anthropic): count consumed provider replays in the timeout ceiling

Codex finding 3786729761 (P1, posted at e443b4f at 19:51:09Z): for a
small request whose first upload received an immediate retryable 529 and
whose provider replay then hit the first-event timeout,
createAnthropicFirstEventTimeoutError derived retryMaxAttempts purely
from request size (2), even though providerRetryAttempt was already 1.
The session counts a whole provider invocation as one attempt, so the
full ceiling licensed a third upload of the same body.

The timeout error now receives providerAttemptsConsumed and reports the
remaining total ceiling (floor 1). Verified live: 529-then-timeout now
reports retryMaxAttempts 1 (was 2); discriminating regression test added
that fails with 2 when the subtraction is removed.

Lore-id: issue-4464-provider-replay-ceiling
Constraint: reported ceiling must bound total uploads across the invocation
Confidence: high
Scope-risk: narrow
Reversibility: trivial
Directive: preserve @probepark attribution for issue #4464
Tested: new test attempts=2 ceiling=1; timeout+iterator suites 31/31 x4; 95/95 session retry suites; both package checks clean after biome fix
Not-tested: managed-fallback accounting interplay beyond existing suites

* fix(anthropic): preserve transport metadata on structured non-Error rejections

Codex finding 3786938429 (P2, posted at cc110df at 20:24:07Z): the
normalizeStreamFailure helper added for primitive-string rejections also
wrapped structured rejections such as { status: 529, error: { type },
headers } from injected clients, collapsing them to Error('[object
Object]') and discarding every transport field — so status/provider-code
extraction and managed-fallback classification lost the failure class.

Structured objects are now wrapped in a mutable Error that copies every
enumerable own property (message via a cycle-safe JSON.stringify with
String fallback); primitives keep the plain String(error) wrapper.
Regression test asserts status 529 + anthropicErrorType survive with
the ceiling facts intact.

Lore-id: issue-4464-structured-rejection-metadata
Constraint: normalization must never drop transport metadata
Confidence: high
Scope-risk: narrow
Reversibility: trivial
Directive: preserve @probepark attribution for issue #4464
Tested: new test passes; timeout+iterator suites 32/32 x4; both package checks clean
Not-tested: getters-only rejection objects

* fix(anthropic): count corrective-policy replays in the timeout ceiling

Codex finding 3787055464 (P2, posted at 2446999 at 20:45:41Z): the
providerAttemptsConsumed fix read providerRetryAttempt, but the four
corrective-policy branches (strict-tool fallback, forced-tool drop,
fast-mode drop, thinking/CPA repair) reset providerRetryAttempt = 0
before continue. A small request that took one corrective replay and
whose corrected upload then timed out still reported the full
two-attempt ceiling despite two uploads already spent, licensing a
third upload of the corrected request.

A separate providerUploadCount counter now tracks total uploads across
the invocation: bumped on every provider retry and every corrective
continue, never reset, and read by the timeout facts. Regression test
covers strict-grammar corrective replay then timeout and asserts the
remaining ceiling is 1 (fails with 2 when the corrective bumps are
removed).

Lore-id: issue-4464-corrective-replay-ceiling
Constraint: ceiling must bound total uploads including corrective replays
Confidence: high
Scope-risk: narrow
Reversibility: trivial
Directive: preserve @probepark attribution for issue #4464
Tested: new test fails without the bumps; timeout+iterator suites 33/33 x4; 95/95 session retry suites; both package checks clean
Not-tested: exotic custom corrective chains beyond the four branches

* fix(anthropic): count thinking and CPA corrective uploads in the timeout ceiling

Codex finding 3787172110 (P2, posted at 84b5378 at 21:08:27Z): the
providerUploadCount counter covered strict-tool, forced-tool, fast-mode,
cache-breakpoint, and generic retries, but not the thinking-replay
repair branch or the CPA alias-repair branch — both corrective
continues that upload a corrected body without resetting the provider
retry budget. A small request whose corrected upload then timed out
still saw count 0 and reported the full two-attempt ceiling, licensing
a third upload.

Both branches now bump providerUploadCount before their replay. All
seven upload sites (initial retries x1, strict/forced/fast/cache x4,
thinking repair, CPA repair) feed the timeout ceiling.

Lore-id: issue-4464-thinking-cpa-upload-count
Constraint: every corrected-body upload must consume the ceiling
Confidence: high
Scope-risk: narrow
Reversibility: trivial
Directive: preserve @probepark attribution for issue #4464
Tested: timeout+iterator suites 33/33 x3; 95/95 session retry suites; packages/ai check clean
Not-tested: dedicated thinking/CPA corrective-then-timeout e2e beyond the shared ceiling path

* test(anthropic): pin thinking and CPA repair uploads against the timeout ceiling

The head commit of this PR marked dedicated thinking/CPA corrective-then-timeout
coverage as Not-tested. Without it, the providerUploadCount increments those
branches added had no direct guard: deleting either one would leave the shared
corrective-policy test green while the ceiling silently licensed a third upload
again — the exact issue #4464 amplification this PR exists to bound.

Both new stubs reproduce the branch-specific rejection first (invalid thinking
signature 400; CPA alias-restore 500), let the repaired body upload, then hang
that upload until the first-event timeout. They assert attempts=2 and the
remaining retryMaxAttempts=1, and both fail with retryMaxAttempts=2 on the
pre-fix parent 84b5378, proving the counter increments are load-bearing.

Lore-id: issue-4464-repair-upload-coverage
Constraint: coverage must fail if any corrective upload site stops counting
Confidence: high
Scope-risk: narrow
Reversibility: trivial
Tested: timeout suite 30/30 x3; repair/cpa/iterator cohorts 44/44; packages/ai check clean
Not-tested: live CPA proxy replay of a steered body

* test(anthropic): isolate corrective timeout replay

The timeout regression used strict-tool fallback even though GJC_NO_STRICT and PI_NO_STRICT intentionally remove strict markers before upload. Reviewers with either supported operator flag therefore saw one upload and a deterministic false failure before the retry-ceiling assertion ran.

Exercise the same resettable corrective-policy path through forced tool-choice fallback, which is independent of strict-schema configuration and still proves the second upload consumes the first-event timeout ceiling.

Lore-id: issue-4464-corrective-test-isolation
Constraint: timeout accounting coverage must pass under supported strict-schema bypass flags
Rejected: change production strict bypass semantics | the operator flags intentionally suppress strict requests
Confidence: high
Scope-risk: narrow
Reversibility: trivial
Tested: anthropic timeout suite 30/30 x5 default, 30/30 x5 GJC_NO_STRICT, 30/30 x5 PI_NO_STRICT
Tested: relevant AI suites 171/171; session retry suites 95/95; packages/ai check clean

* fix(session): preserve snapshot terminalization after rebase

PR #4580 made local snapshot and buffer failures terminal before retry-attempt accounting. Rebasing the timeout-ceiling series retained the obsolete localSnapshot condition from the older control flow, even though that branch is now unreachable.

Match current dev's attempt accounting while retaining the provider retry-ceiling checks added by #4557.

Lore-id: issue-4464-snapshot-rebase-reconciliation
Constraint: local snapshot failures remain terminal and never consume provider fallback attempts
Rejected: retain the obsolete localSnapshot ternary | diverges from current dev after the early terminal return
Confidence: high
Scope-risk: narrow
Reversibility: trivial
Tested: managed attempt transaction and fallback transaction suites 70/70
Tested: agent-session resilient retry and fallback suites 95/95

---------

Co-authored-by: Yeachan Heo <yeachan.heo@gmail.com>
(cherry picked from commit cd51365)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants