Repository navigation
Clarify legacy AI Gateway cutover steps - #1275
Merged
Merged
Conversation
Add what an end-to-end migration to AI Gateway v2 needed in practice: - Attach the AI Gateway route to a Gateway that survives the cutover. Deleting the AIGateway removes its GatewayClass, and disabling the legacy gateway removes its Envoy Gateway controller. - Confirm legacy traffic is gone before deleting the AIGateway, and move platform callers such as the budgets classifier. - Wait for the AIGateway to finish deleting before disabling the legacy gateway, and split the two steps under GitOps. - List the values and resources to remove when disabling the legacy gateway, with the exact condition for keeping directoryGrpcEndpoint. - Keep provider Secrets that a credentialSecretRef still names. - Add troubleshooting for an AIGateway stuck in Terminating. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Alejandro Ponce <aponcedeleonch@stacklok.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
The token-usage check can miss ongoing client requests before gateway deletion and should use request-level evidence.
1 open issue
What changed in this PR
Clarifies the legacy AI Gateway migration guide with cutover sequencing and cleanup steps based on practical migrations.
Changes:
- Adds Gateway ownership requirements and migration steps for platform callers.
- Expands deletion checks, GitOps sequencing, and legacy configuration cleanup.
- Adds recovery instructions for an
AIGatewaystuck in Terminating.
| File | Description |
|---|---|
docs/ai-gateway/migrate-from-legacy.mdx |
Expands cutover prerequisites, cleanup steps, and finalizer troubleshooting. |
Review effort: Balanced
A flat aigw_token_usage_total doesn't prove clients stopped calling the legacy gateway, since rejected requests consume no tokens. The Envoy request counter covers those too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The budgets classifier is an internal experimentation feature and shouldn't appear in published docs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The requirement is that the AI Gateway isn't published through anything the legacy stack owns, whatever ingress the customer uses. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Match the Envoy Gateway prerequisite on the configure page, and explain why the AI Gateway needs its own Envoy Gateway. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drop the platform callers note, since no caller besides the internal classifier needed moving. Scope the request metric check away from Envoy Gateway's readiness and stats listeners, which keep counting from probes and scrapes, and wait for active requests to drain. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| 5. **Delete the `AIGateway` resource** once no traffic reaches it. Do this | ||
| before step 6, while the AI Gateway operator is still installed, so the | ||
| operator cleans up the resources it created. | ||
| 5. **Delete the `AIGateway` resource** once no traffic reaches it. On the legacy |
jerm-dro
approved these changes
Oct 26, 2026
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Updates the legacy AI Gateway migration guide with what an end-to-end migration needed in practice. The gaps came up while removing the legacy gateway from Stacklok's dev and staging clusters (stacklok/infra#6058, stacklok/infra#6059). Everything below was checked against the umbrella chart and the operator source.
httpRouteto a Gateway that survives the cutover. The operator deletes the GatewayClass named inspec.gateway.gatewayClassNamewhen theAIGatewaygoes, even one it didn't create, and disabling the legacy gateway removes its bundled Envoy Gateway controller.enterprise-manager.budgets.classifier.baseURL.aigw_token_usage_total, wait untilkubectl get aigatewayis empty, and reconcile this step on its own under GitOps. The operator's finalizer must run before step 6 removes the operator.directoryGrpcEndpoint(with the exact condition under which the AI Gateway inherits it), operator overrides,gatewayApiUrl, and out-of-chart resources. It also says which CRDs stay. The-api-key-servicesubject has to go in this same change, because the chart guard requires it while the legacy gateway is enabled.providers[].credentialSecretRefstill names.AIGatewaystuck in Terminating.The "Disconnect AI Gateway stops traffic on both" line stays as is. On v2-only installs the console hid that switch, which is a product bug fixed in stacklok/stacklok-enterprise-platform#5650.
Type of change
Related issues/PRs
Submitter checklist
Content and formatting
npm run prettier,npm run eslinton the page, andnpm run buildpass, with no broken links.Reviewer checklist
Content
🤖 Generated with Claude Code