Skip to content

Use the standalone AI gateway in production, staging, and development - #7173

Closed
chrarnoldus wants to merge 4 commits into
mainfrom
add-ai-gateway-route-rewrites
Closed

chrarnoldus wants to merge 4 commits into
mainfrom
add-ai-gateway-route-rewrites

Conversation

@chrarnoldus

@chrarnoldus chrarnoldus commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Route web AI requests to the standalone gateway app while preserving existing client URLs. The rewrites run before web filesystem routes and map gateway/OpenRouter aliases, FIM/edit, System One, and organization model endpoints to /api/v1.

Environment Gateway destination
Production, global and non-global backends https://ai-gateway.kilo.ai
Staging (VERCEL_TARGET_ENV=staging), both backends https://staging-ai-gateway.kilo.ai
Local development Worktree-local gateway
Other preview deployments Existing web handlers

Local development takes precedence over pulled deployment variables, and staging takes precedence over VERCEL_ENV=production. Existing Exa/marketplace routing and unrelated organization/cron endpoints remain unchanged.

Automatic Development Startup

  • pnpm dev:start starts the gateway as a separate dependency of the web app, using worktree-aware ports.
  • pnpm --filter web dev now starts both apps. Its gateway defaults to the selected web port plus 10; AI_GATEWAY_PORT overrides that port.
  • The standalone launcher stops both apps and their workers on Ctrl-C or when either app exits, with bounded cleanup. Managed startup avoids creating a duplicate gateway.
  • Playwright starts and waits for both apps with the same test environment and matching gateway port, supporting Chromatic and local E2E runs.
  • The Next proxy timeout is 800 seconds for long requests. Its application-wide scope is an accepted tradeoff; it does not change Vercel platform limits.

Latest Verification

  • Passed: pnpm exec tsx --test dev/local/web-dev.test.ts dev/local/playwright-gateway.test.ts dev/local/ai-gateway-rewrites.test.ts dev/local/services.test.ts (51 tests).
  • Passed: real pnpm --filter web dev startup with no explicit gateway port, starting web on 5500 and gateway on 5510.
  • Passed: 6 targeted Playwright E2Es against that standalone launch, covering direct gateway startup, FIM/edit and organization-model proxies, a real browser gateway request, and the existing signed-out get-started redirect.
  • Passed: shutdown checks confirming both app ports are released; verification infrastructure was stopped afterward.
  • Passed: web tsgo typecheck, focused oxlint, Bash/Node syntax checks, changed-file formatting, and git diff --check.
  • Earlier checks verified both production backend variants, staging precedence, worktree offsets, and a 31-second delayed proxy response with request data preserved.

Verification Limits

The full repository test suite and authenticated upstream inference were not run. An earlier broader E2E attempt hit two existing signed-in auth-flow timeouts; those tests were not changed. An earlier full dev-tool suite also had a tmux client-attachment failure. These are separate from the passing focused gateway/startup checks.

Rollout

Ensure the production and staging gateway domains point to the corresponding deployments and the gateway project has the equivalent gateway-inference firewall rate-limit rule. Scheduled releases deploy the gateway with the web apps. This PR does not mutate external Vercel configuration.

@chrarnoldus chrarnoldus self-assigned this Oct 5, 2026
Comment thread apps/web/ai-gateway-rewrites.mjs Resolved
Comment thread apps/web/next.config.mjs Resolved
@kilo-code-bot

kilo-code-bot Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: 2 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 1
Issue Details (click to expand)

WARNING

File Line Issue
scripts/web-dev.mjs 70 The web server is spawned detached, leaving the tmux pane's process group. pnpm dev:stop uses tmux kill-session, whose SIGHUP only reaches the pane's foreground group, and this launcher has no SIGHUP handler, so next dev is orphaned and keeps the worktree's port bound.

SUGGESTION

File Line Issue
apps/web/next.config.mjs 68 Global experimental.proxyTimeout also extends the PostHog /ingest/* and global-api.kilo.ai Exa/marketplace proxies to 800s (carried from the previous review; the owner accepted the tradeoff).
Files Reviewed (6 files)
  • scripts/web-dev.mjs - 1 issue
  • scripts/dev.sh - 0 issues
  • dev/local/services.ts - 0 issues
  • dev/local/services.test.ts - 0 issues
  • dev/local/web-dev.test.ts - 0 issues
  • DEVELOPMENT.md - 0 issues

Fix these issues in Kilo Cloud

Previous Review Summaries (5 snapshots, latest commit c8487c2)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit c8487c2)

Status: 2 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 1
Issue Details (click to expand)

WARNING

File Line Issue
apps/web/ai-gateway-rewrites.mjs 4 Development rewrites always target localhost:<AI_GATEWAY_PORT> with no opt-out. Playwright/Chromatic now start the gateway (fixed in c8487c2), but a plain pnpm --filter web dev still starts only the web app, so gateway-backed calls now proxy to a dead port and return 502 instead of the previous in-process handlers.

SUGGESTION

File Line Issue
apps/web/next.config.mjs 68 experimental.proxyTimeout is global; it also extends the PostHog /ingest/* and global-api.kilo.ai Exa/marketplace proxies to 800s.
Files Reviewed (5 files)
  • apps/web/playwright.config.ts - 0 issues
  • apps/web/tests/e2e/ai-gateway-rewrites.spec.ts - 0 issues
  • dev/local/playwright-gateway.test.ts - 0 issues
  • apps/web/ai-gateway-rewrites.mjs - 1 issue (re-verified)
  • apps/web/next.config.mjs - 1 issue (re-verified)

Fix these issues in Kilo Cloud

Previous review (commit 4900696)

Status: 2 Issues Found | Recommendation: Address before merge

Executive Summary

Development-only gateway rewrites assume the standalone AI gateway is always running, so web dev entry points that skip pnpm dev:start (Playwright/Chromatic next dev) now 502 on gateway-backed routes; the global proxyTimeout also widens timeouts for unrelated proxies.

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 1
Issue Details (click to expand)

WARNING

File Line Issue
apps/web/ai-gateway-rewrites.mjs 4 Development rewrites point at localhost:<AI_GATEWAY_PORT> with no opt-out, but Playwright/Chromatic next dev runs (and plain pnpm --filter web dev) do not start the gateway, so gateway-backed calls now 502.

SUGGESTION

File Line Issue
apps/web/next.config.mjs 68 experimental.proxyTimeout is global; it also extends the PostHog /ingest/* and global-api.kilo.ai Exa/marketplace proxies to 800s.
Files Reviewed (7 files)
  • DEVELOPMENT.md - 0 issues
  • apps/ai-gateway/AGENTS.md - 0 issues
  • apps/web/ai-gateway-rewrites.mjs - 1 issue
  • apps/web/next.config.mjs - 1 issue
  • dev/local/ai-gateway-rewrites.test.ts - 0 issues
  • dev/local/services.test.ts - 0 issues
  • dev/local/services.ts - 0 issues

Fix these issues in Kilo Cloud

Previous review (commit 706196f)

Status: 2 Issues Found | Recommendation: Address before merge

Executive Summary

Development-only gateway rewrites assume the standalone AI gateway is always running, so web dev entry points that skip pnpm dev:start (Playwright/Chromatic next dev) now 502 on gateway-backed routes; the global proxyTimeout also widens timeouts for unrelated proxies.

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 1
Issue Details (click to expand)

WARNING

File Line Issue
apps/web/ai-gateway-rewrites.mjs 4 Development rewrites point at localhost:<AI_GATEWAY_PORT> with no opt-out, but Playwright/Chromatic next dev runs (and plain pnpm --filter web dev) do not start the gateway, so gateway-backed calls now 502.

SUGGESTION

File Line Issue
apps/web/next.config.mjs 68 experimental.proxyTimeout is global; it also extends the PostHog /ingest/* and global-api.kilo.ai Exa/marketplace proxies to 800s.
Files Reviewed (7 files)
  • DEVELOPMENT.md - 0 issues
  • apps/ai-gateway/AGENTS.md - 0 issues
  • apps/web/ai-gateway-rewrites.mjs - 1 issue
  • apps/web/next.config.mjs - 1 issue
  • dev/local/ai-gateway-rewrites.test.ts - 0 issues
  • dev/local/services.test.ts - 0 issues
  • dev/local/services.ts - 0 issues

Fix these issues in Kilo Cloud

Previous review (commit d5f420d)

Status: 2 Issues Found | Recommendation: Address before merge

Executive Summary

Development-only gateway rewrites assume the standalone AI gateway is always running, so web dev entry points that skip pnpm dev:start (Playwright/Chromatic next dev) now 502 on gateway-backed routes; the global proxyTimeout also widens timeouts for unrelated proxies.

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 1
Issue Details (click to expand)

WARNING

File Line Issue
apps/web/ai-gateway-rewrites.mjs 4 Development rewrites point at localhost:<AI_GATEWAY_PORT> with no opt-out, but Playwright/Chromatic next dev runs (and plain pnpm --filter web dev) do not start the gateway, so gateway-backed calls now 502.

SUGGESTION

File Line Issue
apps/web/next.config.mjs 68 experimental.proxyTimeout is global; it also extends the PostHog /ingest/* and global-api.kilo.ai Exa/marketplace proxies to 800s.
Files Reviewed (7 files)
  • DEVELOPMENT.md - 0 issues
  • apps/ai-gateway/AGENTS.md - 0 issues
  • apps/web/ai-gateway-rewrites.mjs - 1 issue
  • apps/web/next.config.mjs - 1 issue
  • dev/local/ai-gateway-rewrites.test.ts - 0 issues
  • dev/local/services.test.ts - 0 issues
  • dev/local/services.ts - 0 issues

Fix these issues in Kilo Cloud

Previous review (commit 500eadf)

Status: 2 Issues Found | Recommendation: Address before merge

Executive Summary

Development-only gateway rewrites assume the standalone AI gateway is always running, so web dev entry points that skip pnpm dev:start (Playwright/Chromatic next dev) now 502 on gateway-backed routes; the global proxyTimeout also widens timeouts for unrelated proxies.

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 1
Issue Details (click to expand)

WARNING

File Line Issue
apps/web/ai-gateway-rewrites.mjs 4 Development rewrites point at localhost:<AI_GATEWAY_PORT> with no opt-out, but Playwright/Chromatic next dev runs (and plain pnpm --filter web dev) do not start the gateway, so gateway-backed calls now 502.

SUGGESTION

File Line Issue
apps/web/next.config.mjs 68 experimental.proxyTimeout is global; it also extends the PostHog /ingest/* and global-api.kilo.ai Exa/marketplace proxies to 800s.
Files Reviewed (7 files)
  • DEVELOPMENT.md - 0 issues
  • apps/ai-gateway/AGENTS.md - 0 issues
  • apps/web/ai-gateway-rewrites.mjs - 1 issue
  • apps/web/next.config.mjs - 1 issue
  • dev/local/ai-gateway-rewrites.test.ts - 0 issues
  • dev/local/services.test.ts - 0 issues
  • dev/local/services.ts - 0 issues

Fix these issues in Kilo Cloud


Reviewed by deepseek-v4.1-flash · Input: 92.5K · Output: 32.1K · Cached: 1.3M

Review guidance: REVIEW.md from base branch main

@chrarnoldus
chrarnoldus force-pushed the add-ai-gateway-route-rewrites branch from 500eadf to d5f420d Compare October 7, 2026 14:24
@chrarnoldus chrarnoldus changed the title feat(ai-gateway): rewrite web routes to the standalone gateway feat(ai-gateway): route web traffic to the standalone gateway Oct 7, 2026
@chrarnoldus
chrarnoldus force-pushed the add-ai-gateway-route-rewrites branch from 706196f to 4900696 Compare October 8, 2026 12:28
@chrarnoldus chrarnoldus changed the title feat(ai-gateway): route web traffic to the standalone gateway Use the standalone AI gateway in production, staging, and development Oct 12, 2026
Comment thread scripts/web-dev.mjs
const child = spawn(app.command, app.args, {
cwd: app.cwd,
stdio: 'inherit',
detached: true,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: The web server is spawned detached, so it leaves the tmux pane's process group and escapes dev:stop.

pnpm dev:stop shuts the stack down with tmux kill-session, which only signals the pane's foreground process group. A detached child calls setsid() and lives in its own session, so it never receives that SIGHUP; and because this launcher only registers SIGINT/SIGTERM handlers, a SIGHUP terminates web-dev.mjs without stopping the child. The next dev process is then orphaned (reparented to init), keeps the worktree's web port bound, and makes the next dev:start collide or auto-increment. Previously scripts/dev.sh exec'd next dev inside the pane's group, so kill-session reaped it. Handle SIGHUP as well (e.g. process.on('SIGHUP', () => stop(129))), or avoid detaching and signal each child directly.


Reply with @kilocode-bot fix it to have Kilo Code address this issue.

@chrarnoldus
chrarnoldus deleted the add-ai-gateway-route-rewrites branch October 12, 2026 11:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant