-
Notifications
You must be signed in to change notification settings - Fork 1.4k
fix: set idleTimeout to 255 (Bun max) #659
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
nathanclevenger
wants to merge
26
commits into
anthropics:main
Choose a base branch
from
dot-do:fix/idle-timeout-max-255
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+506
−32
Open
Changes from all commits
Commits
Show all changes
26 commits
Select commit
Hold shift + click to select a range
4d2370c
fix: skip comment update on workflow cancellation
04a053b
chore: sync with upstream anthropics/claude-code-action
nathanclevenger 41123e6
fix: support pull_request.assigned action with track_progress
nathanclevenger 08dd986
feat(bedrock): add AWS_BEARER_TOKEN_BEDROCK support for Bedrock API keys
nathanclevenger 30e8f60
merge: integrate track_progress support for pull_request.assigned events
nathanclevenger 212b1b9
feat: add exponential backoff retry for rate limit errors
nathanclevenger 07cf46a
Merge pull request #1 from dot-do/feature/retry-on-rate-limit
nathanclevenger ce72cb0
feat: fallback to Anthropic API on AWS Bedrock rate limit
nathanclevenger 13f3147
Merge pull request #2 from dot-do/feature/fallback-to-anthropic-on-ra…
nathanclevenger c801b2e
fix: delete progress comment when review is submitted
nathanclevenger a2e32d6
refactor: extract review detection to helper function with type guards
nathanclevenger d4042c6
Merge pull request #3 from dot-do/fix/delete-comment-when-review-subm…
nathanclevenger edb8e53
fix: handle null files data in PR fetcher
nathanclevenger b50aeb0
fix: ALWAYS try Bedrock first on each attempt, immediate Anthropic fa…
nathanclevenger 690c9c2
Merge pull request #5 from dot-do/fix/reduce-anthropic-retry-delay
nathanclevenger 35bb7ac
feat: route Claude API through claude-lb proxy for instant failover
nathanclevenger c566a2c
Merge pull request #6 from dot-do/fix/early-429-detection
nathanclevenger dd4d00c
Merge pull request #4 from dot-do/fix/handle-null-files-pagination
nathanclevenger 029d172
feat: add HTTP proxy for secure pass-through authentication
nathanclevenger d3d06c1
Merge pull request #8 from dot-do/feature/proxy-auth-headers
nathanclevenger 6fc8661
feat: add anthropic-only proxy mode with intelligent fallback (#9)
nathanclevenger aa25fcc
fix: move /health endpoint check before 404 response
nathanclevenger af6557d
fix: move /health endpoint check before 404 response (#10)
nathanclevenger 0cabd93
fix: add idleTimeout and bedrock-only proxy mode (#11)
nathanclevenger a08b95f
refactor: simplify proxy server - require both API keys (#12)
nathanclevenger 61074b1
fix: set idleTimeout to 255 (Bun max) instead of 300
nathanclevenger File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,217 @@ | ||
| /** | ||
| * HTTP Proxy Server for Claude API via Cloudflare AI Gateway | ||
| * | ||
| * Intercepts Claude CLI requests and forwards them directly to Cloudflare AI Gateway, | ||
| * which routes to AWS Bedrock or Anthropic API based on availability. | ||
| * | ||
| * This enables: | ||
| * - Direct routing through AI Gateway (no intermediate worker hop) | ||
| * - Centralized monitoring and observability via Cloudflare AI Gateway | ||
| * - Intelligent failover: Bedrock first (uses $100k credits), then Anthropic | ||
| * - Zero-delay failover on ANY Bedrock error (429, 430, 500, 403, etc.) | ||
| * | ||
| * REQUIREMENTS: | ||
| * - AWS_BEARER_TOKEN_BEDROCK must be set (for Bedrock access) | ||
| * - ANTHROPIC_API_KEY must be set (for failover) | ||
| */ | ||
|
|
||
| const PROXY_PORT = 18765; // Local proxy port | ||
| const AI_GATEWAY_ACCOUNT = 'b6641681fe423910342b9ffa1364c76d'; | ||
| const AI_GATEWAY_ID = 'claude-gateway'; | ||
| const AI_GATEWAY_BASE = `https://gateway.ai.cloudflare.com/v1/${AI_GATEWAY_ACCOUNT}/${AI_GATEWAY_ID}`; | ||
|
|
||
| interface ProxyStats { | ||
| requests: number; | ||
| successes: number; | ||
| failures: number; | ||
| bedrockSuccesses: number; | ||
| anthropicFailovers: number; | ||
| lastError?: string; | ||
| } | ||
|
|
||
| const stats: ProxyStats = { | ||
| requests: 0, | ||
| successes: 0, | ||
| failures: 0, | ||
| bedrockSuccesses: 0, | ||
| anthropicFailovers: 0, | ||
| }; | ||
|
|
||
| /** | ||
| * Validate required credentials are present | ||
| * Throws error if either is missing | ||
| */ | ||
| function validateCredentials(): void { | ||
| const bedrockToken = process.env.AWS_BEARER_TOKEN_BEDROCK?.trim(); | ||
| const anthropicKey = process.env.ANTHROPIC_API_KEY?.trim(); | ||
|
|
||
| const missing: string[] = []; | ||
| if (!bedrockToken) missing.push('AWS_BEARER_TOKEN_BEDROCK'); | ||
| if (!anthropicKey) missing.push('ANTHROPIC_API_KEY'); | ||
|
|
||
| if (missing.length > 0) { | ||
| throw new Error( | ||
| `Missing required credentials: ${missing.join(', ')}\n` + | ||
| 'Both AWS_BEARER_TOKEN_BEDROCK and ANTHROPIC_API_KEY must be set as GitHub org secrets.' | ||
| ); | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * Forward request to AWS Bedrock via AI Gateway | ||
| */ | ||
| async function forwardToBedrock(body: string, headers: Headers): Promise<Response> { | ||
| const bedrockToken = process.env.AWS_BEARER_TOKEN_BEDROCK!; // Validated at startup | ||
|
|
||
| // AI Gateway URL for Bedrock | ||
| const bedrockUrl = `${AI_GATEWAY_BASE}/aws-bedrock/bedrock-runtime.us-east-1.amazonaws.com/model/us.anthropic.claude-sonnet-4-5-v1:0/invoke`; | ||
|
|
||
| const bedrockHeaders = new Headers({ | ||
| 'Content-Type': 'application/json', | ||
| 'Authorization': `Bearer ${bedrockToken}`, | ||
| 'anthropic-version': headers.get('anthropic-version') || '2023-06-01', | ||
| }); | ||
|
|
||
| console.log('🔵 Attempting Bedrock via AI Gateway...'); | ||
|
|
||
| const response = await fetch(bedrockUrl, { | ||
| method: 'POST', | ||
| headers: bedrockHeaders, | ||
| body | ||
| }); | ||
|
|
||
| if (response.ok) { | ||
| console.log('✅ Bedrock request succeeded via AI Gateway'); | ||
| } else { | ||
| console.warn(`⚠️ Bedrock returned ${response.status}`); | ||
| } | ||
|
|
||
| return response; | ||
| } | ||
|
|
||
| /** | ||
| * Forward request to Anthropic via AI Gateway | ||
| */ | ||
| async function forwardToAnthropic(body: string, headers: Headers): Promise<Response> { | ||
| const anthropicKey = process.env.ANTHROPIC_API_KEY!; // Validated at startup | ||
|
|
||
| // AI Gateway URL for Anthropic | ||
| const anthropicUrl = `${AI_GATEWAY_BASE}/anthropic/v1/messages`; | ||
|
|
||
| const anthropicHeaders = new Headers({ | ||
| 'Content-Type': 'application/json', | ||
| 'x-api-key': anthropicKey, | ||
| 'anthropic-version': headers.get('anthropic-version') || '2023-06-01', | ||
| }); | ||
|
|
||
| console.log('🟢 Attempting Anthropic via AI Gateway...'); | ||
|
|
||
| const response = await fetch(anthropicUrl, { | ||
| method: 'POST', | ||
| headers: anthropicHeaders, | ||
| body | ||
| }); | ||
|
|
||
| if (response.ok) { | ||
| console.log('✅ Anthropic request succeeded via AI Gateway'); | ||
| } else { | ||
| console.error(`❌ Anthropic returned ${response.status}`); | ||
| } | ||
|
|
||
| return response; | ||
| } | ||
|
|
||
| export async function startProxyServer(): Promise<number> { | ||
| // Validate credentials at startup - fail fast if misconfigured | ||
| validateCredentials(); | ||
|
|
||
| const server = Bun.serve({ | ||
| port: PROXY_PORT, | ||
| hostname: '127.0.0.1', | ||
| idleTimeout: 255, // Max allowed by Bun (4.25 minutes) for long-running Claude API requests | ||
|
|
||
| async fetch(req: Request): Promise<Response> { | ||
| const url = new URL(req.url); | ||
|
|
||
| // Health check endpoint | ||
| if (url.pathname === '/health') { | ||
| return Response.json(stats); | ||
| } | ||
|
|
||
| // Only proxy /v1/messages | ||
| if (url.pathname !== '/v1/messages' || req.method !== 'POST') { | ||
| return new Response('Not Found', { status: 404 }); | ||
| } | ||
|
|
||
| stats.requests++; | ||
|
|
||
| const body = await req.text(); | ||
|
|
||
| // Always try Bedrock first (uses $100k credits) | ||
| try { | ||
| const bedrockResponse = await forwardToBedrock(body, req.headers); | ||
|
|
||
| if (bedrockResponse.ok) { | ||
| stats.successes++; | ||
| stats.bedrockSuccesses++; | ||
| return bedrockResponse; | ||
| } | ||
|
|
||
| // Any Bedrock error triggers immediate failover to Anthropic | ||
| console.log(`⚠️ Bedrock returned ${bedrockResponse.status} - failing over to Anthropic`); | ||
| } catch (error) { | ||
| console.error('❌ Bedrock request error:', error); | ||
| } | ||
|
|
||
| // Immediate failover to Anthropic (zero delay) | ||
| console.log('🔄 Failing over to Anthropic...'); | ||
| stats.anthropicFailovers++; | ||
|
|
||
| try { | ||
| const anthropicResponse = await forwardToAnthropic(body, req.headers); | ||
|
|
||
| if (anthropicResponse.ok) { | ||
| stats.successes++; | ||
| } else { | ||
| stats.failures++; | ||
| stats.lastError = `Anthropic returned ${anthropicResponse.status}`; | ||
| } | ||
|
|
||
| return anthropicResponse; | ||
| } catch (error) { | ||
| console.error('❌ Anthropic request error:', error); | ||
| stats.failures++; | ||
| stats.lastError = String(error); | ||
|
|
||
| return new Response( | ||
| JSON.stringify({ | ||
| error: 'Both Bedrock and Anthropic failed', | ||
| message: String(error) | ||
| }), | ||
| { status: 500, headers: { 'Content-Type': 'application/json' } } | ||
| ); | ||
| } | ||
| } | ||
| }); | ||
|
|
||
| console.log(`✅ Proxy server listening on http://127.0.0.1:${server.port}`); | ||
| console.log(` Forwarding to Cloudflare AI Gateway (${AI_GATEWAY_BASE})`); | ||
| console.log(' 🔒 Bedrock-first with Anthropic failover'); | ||
| console.log(` 📊 Health endpoint: http://127.0.0.1:${server.port}/health`); | ||
|
|
||
| return server.port; | ||
| } | ||
|
|
||
| export function getProxyUrl(): string { | ||
| return `http://127.0.0.1:${PROXY_PORT}`; | ||
| } | ||
|
|
||
| export function shouldUseProxy(): boolean { | ||
| // Proxy should be used if both credentials are available | ||
| try { | ||
| validateCredentials(); | ||
| return true; | ||
| } catch { | ||
| return false; | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
idleTimeout: 0will make it unlimited