Remove the InferenceGateway feature flag and enable the gateway everywhere#473
Merged
Conversation
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The inference gateway has been running behind the
InferenceGatewayfeature flag since it shipped. This removes the flag and makes the gateway the unconditional behavior for every deployment: gateway-covered provider API keys and ChatGPT subscription auth always stay on the control plane, and task sandboxes route inference through/api/inferencewith their run-scoped token.Changes
InferenceGatewayenum entry, its config block, theisInferenceGatewayEnabledForDeploymentfail-closed evaluator helper, and the server export.inferenceGatewayboolean option is gone from the resolver's public API.resolveEffectiveModelRuntimeEnvis now the control-plane entry point (routing, titles, summaries — no run token, so raw provider keys), and a newresolveSandboxModelRuntimeEnvis the sandbox entry point (gateway-covered keys stay on the control plane, advertised viaR_INFERENCE_GATEWAY_KEYS; ChatGPT subscriptions route through the gateway).fetchResolvedRuntimeEnvVarscalls the sandbox variant; the shared implementation keeps the distinction as a private parameter.inferenceGatewayEnabledoption fromBuildWorkerEnvOptionsand all five worker-env builders;buildBaseWorkerEnvnow always withholds gateway-covered provider keys from the worker daemon env. Deleted the controller'sinference-gateway-flag.tshelper and its use in the five spawn-worker paths.mockFeatureFlags.OPENROUTER_API_KEY,AI_GATEWAY_API_KEY) are asserted withheld while custom provider keys still flow through.Notes
inference_gatewayin their deployment metadata will see it surface as a legacy metadata control in the admin UI; it is ignored and can be deleted.Validation
pnpm check-types,pnpm lint,pnpm knipall pass.