Skip to content

feat: inject OpenRouter provider.only for BYOK / no-fallback routing - #19

Open
echarrod wants to merge 1 commit into
nielspeter:mainfrom
echarrod:feat/openrouter-provider-only
Open

feat: inject OpenRouter provider.only for BYOK / no-fallback routing#19
echarrod wants to merge 1 commit into
nielspeter:mainfrom
echarrod:feat/openrouter-provider-only

Conversation

@echarrod

Copy link
Copy Markdown

What

Adds an OPENROUTER_PROVIDER_ONLY env var (comma-separated provider slugs) that the proxy injects as {"provider": {"only": [...]}} into every request body when the detected provider is OpenRouter. This pins routing to specific upstream provider(s) so requests use the user's BYOK key and fail hard instead of silently falling back to a credit-billed provider (e.g. Anthropic → Bedrock, Google → Vertex).

Closes #8 (and the feature requested in #18).

Why

When using OpenRouter with a BYOK key, OpenRouter can fall back to a different provider serving the same model if the primary provider errors or is rate-limited. That fallback bills OpenRouter credits even when the user only wants to use their own provider key. OpenRouter's documented fix is to send a provider field in the body, but the proxy had no way to configure it. The proxy already builds and marshals the OpenAI request body in internal/converter/converter.go, so it is the natural place to inject this.

Changes

  • pkg/models/types.go — add Provider map[string]interface{} json:"provider,omitempty" to OpenAIRequest.
  • internal/config/config.go — add OpenRouterProviderOnly []string, parsed from OPENROUTER_PROVIDER_ONLY (trimmed, empties dropped, nil when unset).
  • internal/converter/converter.go — in the OpenRouter path, set openaiReq.Provider = {"only": cfg.OpenRouterProviderOnly} when configured. Applied to both streaming and non-streaming requests (not gated on the streaming branch).

Design notes

  • allow_fallback is intentionally omitted. OpenRouter returns 400 provider: Unrecognized key: "allow_fallback" if it is included inside provider, and only already implies fail-hard (no fallback) per OpenRouter's docs.
  • Provider pinning constrains the provider slug only. It does not prevent within-slug fallback from a BYOK endpoint to OpenRouter's own pool for the same slug when the BYOK endpoint errors (e.g. 429). That still requires the "Always use for this provider" toggle on a Prioritized BYOK key in OpenRouter's workspace settings. I've verified this caveat live (see below) and think it's worth documenting in the README alongside the feature so users don't assume provider.only alone gives them zero credit usage.

Testing

  • go build clean; go test ./... all packages pass (config, converter, daemon, server).
  • Verified end-to-end against OpenRouter with OPENROUTER_PROVIDER_ONLY=google-ai-studio and ANTHROPIC_DEFAULT_OPUS_MODEL=google/gemini-2.5-pro:
    • Debug log confirms the outgoing request body contains "provider": {"only": ["google-ai-studio"]}.
    • OpenRouter's generation metadata confirms the request routed to Google AI Studio (no Bedrock/Vertex cross-provider fallback) — the slug pin works as intended.
    • Confirmed the within-slug caveat above: when the BYOK endpoint returned 429, OpenRouter fell back to its own google-ai-studio endpoint (is_byok: false) within the same slug and billed credits. provider.only blocked the cross-provider path but could not block this; the BYOK toggle is required for that.

Usage

In ~/.claude/proxy.env (or .env):

OPENROUTER_PROVIDER_ONLY=google-ai-studio

Omit the var to leave routing behaviour unchanged.

Add OPENROUTER_PROVIDER_ONLY env var (comma-separated provider slugs)
that the proxy injects as {"provider": {"only": [...]}} into every
request body when the detected provider is OpenRouter. This pins
routing to specific upstream provider(s) so requests use the user's
BYOK key and fail hard instead of silently falling back to a
credit-billed provider (e.g. Bedrock/Vertex).

allow_fallback is intentionally omitted: OpenRouter rejects it as an
unrecognized key inside provider, and only already implies fail-hard.

Note: provider.only constrains the provider slug only. It does not
prevent within-slug fallback from a BYOK endpoint to OpenRouter's own
pool for the same slug; that still requires the "Always use for this
provider" toggle on a Prioritized BYOK key.

Closes nielspeter#8

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature Request: Provider routing for OpenRouter

1 participant