Skip to content
Open
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@ cleans up after itself.
the models included in your subscription.
2. **API keys (BYOK).** Paste a key from OpenRouter, Anthropic, OpenAI, xAI,
Google Gemini, Amazon Bedrock, Vercel AI Gateway,
Cloudflare AI Gateway, Cloudflare Workers AI,
Baseten, Together AI, Moonshot AI (Kimi), Kimi for Coding, MiniMax,
Z.AI (including Coding Plan), OpenCode Zen / Go, or GitHub Copilot.

Expand Down Expand Up @@ -232,7 +233,8 @@ it runs.
**What models does it support?**
Two options. Connect your ChatGPT Plus or Pro subscription directly (no API key
needed), or paste an API key from OpenRouter, Anthropic, OpenAI, xAI, Google
Gemini, Amazon Bedrock, Vercel AI Gateway, Baseten,
Gemini, Amazon Bedrock, Vercel AI Gateway, Cloudflare AI Gateway,
Cloudflare Workers AI, Baseten,
Together AI, Moonshot AI (Kimi), Kimi for Coding, MiniMax, Z.AI (including
Coding Plan), OpenCode Zen / Go, or GitHub Copilot.

Expand Down
5 changes: 5 additions & 0 deletions SELF_HOSTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -401,6 +401,11 @@ common provider keys into worker containers:

- `OPENROUTER_API_KEY`
- `AI_GATEWAY_API_KEY` (Vercel AI Gateway, `vercel/...` models)
- `CLOUDFLARE_AI_GATEWAY_API_TOKEN`, `CLOUDFLARE_AI_GATEWAY_ACCOUNT_ID`,
and `CLOUDFLARE_AI_GATEWAY_ID` (Cloudflare AI Gateway,
`cloudflare-ai-gateway/...` models)
- `CLOUDFLARE_WORKERS_AI_API_TOKEN` and `CLOUDFLARE_WORKERS_AI_ACCOUNT_ID`
(Cloudflare Workers AI, `cloudflare-workers-ai/...` models)
- `OPENAI_API_KEY`
- `ANTHROPIC_API_KEY`
- `MOONSHOT_API_KEY`
Expand Down
103 changes: 103 additions & 0 deletions apps/api/src/handlers/inference/__tests__/inference-gateway.test.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

40 changes: 37 additions & 3 deletions apps/api/src/handlers/inference/registry.ts
Original file line number Diff line number Diff line change
Expand Up @@ -79,23 +79,57 @@ export async function resolveGatewayUpstream(
};
}

const requiredHeaders = await resolveRequiredForwardHeaders(provider);

return {
ok: true,
resolved: {
upstreamUrl: `${upstreamBaseUrl}${upstreamPath}${search}`,
headers:
apiKey && provider.authHeader
headers: {
...requiredHeaders,
...(apiKey && provider.authHeader
? {
[provider.authHeader.name]: formatProviderAuthHeaderValue(
provider,
apiKey,
),
}
: {},
: {}),
},
},
};
}

async function resolveRequiredForwardHeaders(
provider: InferenceGatewayProvider,
): Promise<Record<string, string>> {
if (!provider.requiredHeaders?.length) {
return {};
}

const headers: Record<string, string> = {};

for (const spec of provider.requiredHeaders) {
const value = await resolveModelProviderEnvValue([spec.envVarName]);

if (!value) {
throw new Error(
`${spec.envVarName} must be configured for ${provider.name}.`,
);
}

if (!INFERENCE_GATEWAY_RESOURCE_PATTERN.test(value)) {
throw new Error(
`${spec.envVarName} must be a valid resource name for ${provider.name}. Received "${value}".`,
);
}

headers[spec.headerName] = value;
}

return headers;
}

/**
* xAI supports both SuperGrok OAuth and a BYOK API key. Prefer a connected
* subscription (fresh access token) so subscription users never need a key;
Expand Down
2 changes: 2 additions & 0 deletions apps/docs/docs.json
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,8 @@
"providers/inference/azure-openai",
"providers/inference/baseten",
"providers/inference/chatgpt",
"providers/inference/cloudflare-ai-gateway",
"providers/inference/cloudflare-workers-ai",
"providers/inference/github-copilot",
"providers/inference/google-gemini",
"providers/inference/kimi-for-coding",
Expand Down
5 changes: 5 additions & 0 deletions apps/docs/environment-variables.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -188,6 +188,11 @@ as per-task auth tokens or workspace paths.
| `AI_GATEWAY_API_KEY` | Provider key | Vercel AI Gateway API key. |
| `BASETEN_API_KEY` | Provider key | Baseten API key. |
| `TOGETHER_API_KEY` | Provider key | Together AI API key. |
| `CLOUDFLARE_AI_GATEWAY_API_TOKEN` | Provider key | Cloudflare AI Gateway API token. Does not connect Workers AI. |
| `CLOUDFLARE_AI_GATEWAY_ACCOUNT_ID` | Provider config | Cloudflare account ID for AI Gateway requests. |
| `CLOUDFLARE_AI_GATEWAY_ID` | Provider config | Cloudflare AI Gateway ID, for example `default`. |
| `CLOUDFLARE_WORKERS_AI_API_TOKEN` | Provider key | Cloudflare Workers AI API token. Does not connect AI Gateway. |
| `CLOUDFLARE_WORKERS_AI_ACCOUNT_ID` | Provider config | Cloudflare account ID for Workers AI requests. A gateway ID is not used. |
| `OPENAI_API_KEY` | Provider key | OpenAI API key. |
| `AZURE_API_KEY` | Provider key | Azure OpenAI API key. |
| `AZURE_RESOURCE_NAME` | Provider config | Azure OpenAI resource name, without the domain or URL. |
Expand Down
2 changes: 2 additions & 0 deletions apps/docs/models.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,8 @@ These connections use metered API billing or a provider-managed gateway:
| [Azure AI Foundry](/providers/inference/azure-foundry) | Azure AI Services API key and resource name | Azure subscription |
| [Azure OpenAI](/providers/inference/azure-openai) | Azure OpenAI API key and resource name | Azure subscription |
| [Baseten](/providers/inference/baseten) | Baseten API key | Baseten workspace |
| [Cloudflare AI Gateway](/providers/inference/cloudflare-ai-gateway) | Cloudflare API token, account ID, and gateway ID | Cloudflare account |
| [Cloudflare Workers AI](/providers/inference/cloudflare-workers-ai) | Cloudflare API token and account ID | Cloudflare account |
| [Google Gemini](/providers/inference/google-gemini) | Google AI Studio key | Google Cloud project |
| [MiniMax](/providers/inference/minimax) | MiniMax API key | MiniMax account |
| [Moonshot AI (Kimi)](/providers/inference/moonshot-ai) | Kimi Open Platform key | Moonshot Open Platform balance |
Expand Down
67 changes: 67 additions & 0 deletions apps/docs/providers/inference/cloudflare-ai-gateway.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
---
title: Cloudflare AI Gateway
icon: 'https://unpkg.com/@lobehub/icons-static-svg@1.94.0/icons/cloudflare.svg'
description: Route Roomote model calls through Cloudflare AI Gateway.
---

Cloudflare AI Gateway is a multi-vendor control plane for model calls. Use it
when you want one Cloudflare token, account, and gateway to reach models from
several providers, with Cloudflare's logging, caching, and routing in front.

This is a separate connection from [Cloudflare Workers AI](/providers/inference/cloudflare-workers-ai).
Connecting AI Gateway does not connect Workers AI.

## Get credentials

Create a [Cloudflare API token](https://developers.cloudflare.com/fundamentals/api/get-started/create-token/)
with AI Gateway access. Copy the account ID from the Cloudflare dashboard, then
create or select an [AI Gateway](https://developers.cloudflare.com/ai-gateway/get-started/)
and copy its gateway ID. A `default` gateway is created automatically on first
use.

Configure stored provider keys or unified billing in Cloudflare for the vendors
you plan to call through the gateway.

## Configuration

Add **Cloudflare AI Gateway** in **Settings > Models**, then enter the API
token, account ID, and gateway ID. Environment-variable configuration uses:

```sh
CLOUDFLARE_AI_GATEWAY_API_TOKEN=...
CLOUDFLARE_AI_GATEWAY_ACCOUNT_ID=your-account-id
CLOUDFLARE_AI_GATEWAY_ID=default
```

Roomote exposes supported models under the
`cloudflare-ai-gateway/<vendor>/<model>` prefix and adds a recommended
cross-vendor set. Apply the recommended mapping or choose models individually
for each role.

In gateway mode, the API token stays on the Roomote control plane. Requests are
proxied to Cloudflare's OpenAI-compatible `/ai/v1` surface with the account ID
in the URL and the gateway ID in the `cf-aig-gateway-id` header.

## Cost behavior

Cloudflare records gateway usage and applies the billing terms, stored-key
routing, and upstream provider pricing configured for the account. Roomote
records token usage and estimates model cost from metadata; Cloudflare usage
and invoice are authoritative. See [AI Gateway pricing](https://developers.cloudflare.com/ai-gateway/pricing/).

## Verify setup

1. save the API token, account ID, and gateway ID
2. confirm Cloudflare AI Gateway models appear in **Settings > Models**
3. enable a model and assign it to the coding role
4. run a small task and confirm it appears in Cloudflare AI Gateway logs

## Common issues

- **The token is rejected.** Confirm it is a Cloudflare API token with AI
Gateway permission, not an unrelated Cloudflare global API key.
- **A model cannot be routed.** Check that the vendor is enabled on the
selected gateway and that stored keys or unified billing cover that model.
- **Workers AI models fail.** `@cf/` models through AI Gateway still need
Workers AI access on the token. Connecting Workers AI as its own provider is
a separate step.
67 changes: 67 additions & 0 deletions apps/docs/providers/inference/cloudflare-workers-ai.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
---
title: Cloudflare Workers AI
icon: 'https://unpkg.com/@lobehub/icons-static-svg@1.94.0/icons/cloudflare.svg'
description: Use Cloudflare-hosted Workers AI models for Roomote tasks.
---

Cloudflare Workers AI hosts open and specialized models on Cloudflare's
network. Roomote's direct provider exposes a curated `@cf/` subset suited to
coding and agent work.

This is a separate connection from [Cloudflare AI Gateway](/providers/inference/cloudflare-ai-gateway).
Connecting Workers AI does not require a gateway ID and does not connect AI
Gateway.

## Get credentials

Create a [Cloudflare API token](https://developers.cloudflare.com/fundamentals/api/get-started/create-token/)
with Workers AI access. Copy the account ID from the Cloudflare dashboard. A
gateway ID is not used for this provider.

Make sure the account has Workers AI enabled for the models you select. See
the [Workers AI model catalog](https://developers.cloudflare.com/workers-ai/models/).

## Configuration

Add **Cloudflare Workers AI** in **Settings > Models**, then enter the API
token and account ID. Environment-variable configuration uses:

```sh
CLOUDFLARE_WORKERS_AI_API_TOKEN=...
CLOUDFLARE_WORKERS_AI_ACCOUNT_ID=your-account-id
```

Roomote adds supported models with the `cloudflare-workers-ai/` prefix, using
Cloudflare-hosted `@cf/` catalog IDs. Enable the models you want and assign the
default coding and specialized roles. Provider model names are case-sensitive,
so use the IDs shown in Roomote rather than typing a similar slug from another
gateway.

In gateway mode, the API token stays on the Roomote control plane. Requests are
proxied to Cloudflare's OpenAI-compatible
`/accounts/<account>/ai/v1` surface. No gateway header is sent.

## Cost behavior

Cloudflare charges the connected account according to the selected Workers AI
model. Roomote records usage and estimates cost when model metadata includes
pricing; Cloudflare billing and the [Workers AI pricing page](https://developers.cloudflare.com/workers-ai/platform/pricing/)
are authoritative.

## Verify setup

1. save the API token and account ID
2. confirm Workers AI models appear in **Settings > Models**
3. enable one `@cf/` model and assign it as the coding model
4. run a small task and confirm the request appears in Workers AI usage

## Common issues

- **Authentication fails.** Confirm the token has Workers AI permission and
belongs to the configured account.
- **A model ID is rejected.** Use the exact `cloudflare-workers-ai/@cf/...` ID
shown in Roomote. IDs from AI Gateway or another provider are not
interchangeable.
- **Requests mention a missing gateway.** You are calling the AI Gateway
provider, not Workers AI. Connect **Cloudflare Workers AI** when you want
hosted `@cf/` models without a gateway ID.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions apps/web/src/app/(onboarding)/setup/setup-docs.ts
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,8 @@ const MODEL_PROVIDER_DOC_PATHS: Partial<
'azure-cognitive-services': 'providers/inference/azure-foundry',
baseten: 'providers/inference/baseten',
chatgpt: 'providers/inference/chatgpt',
'cloudflare-ai-gateway': 'providers/inference/cloudflare-ai-gateway',
'cloudflare-workers-ai': 'providers/inference/cloudflare-workers-ai',
'github-copilot': 'providers/inference/github-copilot',
google: 'providers/inference/google-gemini',
'kimi-for-coding': 'providers/inference/kimi-for-coding',
Expand Down
Loading
Loading