feat(runtime-fallback): auto-retry, generic retry detection, and timeout toggle

[youngbinkim0]

Summary

This PR implements runtime fallback auto-retry with full agent preservation. The fix chain enables end-to-end runtime fallback to work correctly in production.

What Changed

Bug Fixes (9 total)

1. Nested status code detection - Anthropic errors have data.statusCode structure
2. Regex pattern relaxed - Match "credit balance too low" with flexible spacing
3. Schema limit bumped - max_fallback_attempts now supports 20 (was 10)
4. Fallback models discovery - Handle sparse session.error events without agent info
5. Model fallback init - Derive model from agent config when events lack data
6. Auto-retry with promptAsync - Resend last user message after fallback selection
7. Persistent fallback - Fallback model now persists across messages in session
8. Manual model override - Detect user-initiated model changes and reset fallback state
9. Agent preservation - Resolve correct agent for auto-retry (was defaulting to sisyphus)

New Features

• Provider-agnostic retry detection - Generalized pattern for Copilot, OpenAI, and future providers
• Timeout toggle - timeout_seconds: 0 disables quota-based fallback escalation
• Agent/category fallback_models - Support custom fallback chains per agent/category
• Provider-aware model resolution - Check connectivity before matching fallback models

Testing

✅ 44 runtime-fallback tests passing
✅ 46 schema tests passing
✅ TypeScript compilation clean
✅ Manual end-to-end verification (Anthropic error → auto-fallback → agent preserved → manual override works)

Related

Builds on #1408 (feat/runtime-fallback-only)

[github-actions]

All contributors have signed the CLA. Thank you! ✅
_{Posted by the CLA Assistant Lite bot.}

[youngbinkim0]

I have read the CLA Document and I hereby sign the CLA

[youngbinkim0]

recheck

[youngbinkim0]

recheck

[cubic-dev-ai]

2 issues found across 27 files

Confidence score: 3/5

• Potential user-impacting inconsistency in src/shared/model-resolution-pipeline.ts: new userFallbackModels reads connected providers only from cache, ignoring constraints.connectedProviders, which could change model resolution behavior.
• Code duplication in src/hooks/runtime-fallback/index.ts auto-retry handlers raises maintenance risk and could lead to subtle divergence, though it’s not an immediate blocker.
• Pay close attention to src/shared/model-resolution-pipeline.ts and src/hooks/runtime-fallback/index.ts - connected provider handling and duplicated retry logic.

Prompt for AI agents (all issues)

Check if these issues are valid — if so, understand the root cause of each and fix them.


<file name="src/hooks/runtime-fallback/index.ts">

<violation number="1" location="src/hooks/runtime-fallback/index.ts:473">
P2: Significant code duplication in auto-retry logic between `session.error` and `message.updated` event handlers. The logic for fetching session messages, extracting the last user message, parsing parts, and calling `promptAsync` is duplicated almost verbatim (approximately 40+ lines). This increases maintenance burden and risk of inconsistencies.</violation>
</file>

<file name="src/shared/model-resolution-pipeline.ts">

<violation number="1" location="src/shared/model-resolution-pipeline.ts:105">
P1: Inconsistent handling of `constraints.connectedProviders` - the new `userFallbackModels` logic reads connected providers only from cache, ignoring the `constraints.connectedProviders` parameter that is respected in `categoryDefaultModel` and `fallbackChain` logic. This prevents callers from enforcing specific provider constraints in the user fallback code path.</violation>
</file>

---

Since this is your first cubic review, here's how it works:

• cubic automatically reviews your code and comments on bugs and improvements
• Teach cubic by replying to its comments. cubic learns from your replies and gets better over time
• Add one-off context when rerunning by tagging @cubic-dev-ai with guidance or docs links (including llms.txt)
• Ask questions if you need clarification on any suggestion

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[youngbinkim0]

Summary

This PR implements runtime fallback improvements to detect provider auto-retry signals and adds a timeout toggle for quota-based fallback escalation.

Changes

1. Generalized Provider Auto-Retry Detection

• File: src/hooks/runtime-fallback/index.ts
• Replaced hardcoded provider-specific checks with unified extractAutoRetrySignal() function
• Detects ANY provider message containing "limit/quota/rate" keywords AND "[retrying in X]"
• Extensible: add new patterns to PROVIDER_RETRY_PATTERNS array
• Works for GitHub Copilot, OpenAI, and future providers

2. Timeout Toggle for Quota Detection

• Files: src/hooks/runtime-fallback/index.ts, src/config/schema/runtime-fallback.ts, docs/configurations.md
• When timeout_seconds: 0, auto-retry signal detection is disabled
• Error-based fallback (429, 503, etc.) continues to work normally
• Allows users to disable timeout-based escalation while keeping error fallback

3. OpenAI Retry Signal Support

• Files: src/hooks/runtime-fallback/constants.ts, src/hooks/runtime-fallback/index.test.ts
• Added OpenAI pattern: /usage\s+limit\s+has\s+been\s+reached/i
• Tests verify OpenAI retry signal triggers fallback escalation
  How It Works
  When a provider (Copilot/OpenAI) emits a retry message like:

"The usage limit has been reached [retrying in 27s attempt #6]"
The hook now:

1. Detects the auto-retry signal pattern
2. Logs: "Detected provider auto-retry signal"
3. Keeps fallback timeout armed (doesn't clear it on false success)
4. When timeout fires, aborts current request and advances to next fallback model

Test Results

• ✅ 44 runtime-fallback tests passing
• ✅ 46 schema tests passing
• ✅ TypeScript compilation clean
• ✅ Build successful

Example Configuration

{
"runtime_fallback": {
"enabled": true,
"retry_on_errors": [429, 503, 529],
"max_fallback_attempts": 10,
"cooldown_seconds": 60,
"timeout_seconds": 0,
"notify_on_fallback": true
}
}
With timeout_seconds: 0, only actual errors (429, 503, etc.) trigger fallback - no timeout-based escalation.