Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: expand analytics support in README and code for Cloudflare AI G… #58

Merged
merged 2 commits into from
Aug 23, 2024
Merged

feat: expand analytics support in README and code for Cloudflare AI G… #58

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0e990e0
feat: expand analytics support in README and code for Cloudflare AI G…
umutbozdag Aug 16, 2024
40e5a0a
fix: change typo in authorization header for cloudflare gateway
ogzhanolguncu Aug 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Features:
- (Optional) built-in Redis compatibility for fast chat history management
- (Optional) built-in rate limiting
- (Optional) disableRag option to use it as LLM + chat history
- (Optional) Analytics via [Helicone](https://www.helicone.ai/) and [Langsmith](https://www.langchain.com/langsmith)
- (Optional) Analytics via [Helicone](https://www.helicone.ai/), [Langsmith](https://www.langchain.com/langsmith) and [Cloudflare AI Gateway](https://developers.cloudflare.com/ai-gateway/)

## Getting started

Expand Down
20 changes: 20 additions & 0 deletions src/models.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,26 @@
});
});

test("OpenAI client configuration with Cloudflare AI Gateway", () => {
const client = openai("gpt-3.5-turbo", {
analytics: {
name: "cloudflare",
accountId: "mock-account-id",
gatewayName: "mock-gateway-name",
},
});

//@ts-expect-error required for testing
expect(client.clientConfig.baseURL).toBe(

Check failure on line 62 in src/models.test.ts

View workflow job for this annotation

GitHub Actions / Tests

error: expect(received).toBe(expected) 

Expected: "https://gateway.ai.cloudflare.com/v1/mock-account-id/mock-gateway-name"
Received: "https://gateway.ai.cloudflare.com/v1/mock-account-id/mock-gateway-name/openai"

      at /home/runner/work/rag-chat/rag-chat/src/models.test.ts:62:41
"https://gateway.ai.cloudflare.com/v1/mock-account-id/mock-gateway-name"
);
//@ts-expect-error required for testing
expect(client.clientConfig.defaultHeaders).toEqual({
Authorization: "Bearer mock-cloudflare-token",
"Content-Type": "application/json",
});
});

test("OpenAI client configuration without analytics", () => {
const config = {
apiKey: "no-key",
Expand Down
13 changes: 12 additions & 1 deletion src/models.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,8 @@ type Providers =
| "mistral";
type AnalyticsConfig =
| { name: "helicone"; token: string }
| { name: "langsmith"; token: string; apiUrl?: string };
| { name: "langsmith"; token: string; apiUrl?: string }
| { name: "cloudflare"; accountId: string; gatewayName: string };

type ModelOptions = Omit<LLMClientConfig, "model"> & {
analytics?: AnalyticsConfig;
Expand Down Expand Up @@ -138,6 +139,16 @@ const setupAnalytics = (
}
return { client: undefined };
}
case "cloudflare": {
return {
baseURL: `https://gateway.ai.cloudflare.com/v1/${analytics.accountId}/${analytics.gatewayName}/openai`,
defaultHeaders: {
Authorization: `Bearer ${providerApiKey}`,
"Content-Type": "application/json",
},
};
}

default: {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
throw new Error(`Unsupported analytics provider: ${JSON.stringify(analytics)}`);
Expand Down
Loading