Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Put a language model in a web app without running an inference backend. It runs


## No API Key or BYOK
On-device models don't need API keys. Cloud models use BYOK (bring your own key): each user supplies their own, and it stays on the client instead of on a server you run.
On-device models don't need API keys. Most cloud providers use BYOK (bring your own key): each user supplies their own, and it stays on the client instead of on a server you run. Firebase AI Logic instead uses the host app's configured Firebase project, backend, Authentication, and App Check context.

## Run a model in the browser

Expand Down Expand Up @@ -47,7 +47,7 @@ for await (const ev of client.chat({
}
```

The weights download once via Transformers.js and run on WebGPU (WASM fallback when there's no GPU). `createEngineModelClient` wraps the engine as a `ModelClient` — the same interface every cloud provider implements, so swapping it for `geminiModelClient({ apiKey, model })` (or `openrouterModelClient`, `requestyModelClient`, `ollamaModelClient`, …) changes that one line and nothing else.
The weights download once via Transformers.js and run on WebGPU (WASM fallback when there's no GPU). `createEngineModelClient` wraps the engine as a `ModelClient` — the same interface every cloud provider implements, so swapping it for `geminiModelClient({ apiKey, model })`, `createFirebaseAiLogicModelClient(firebaseModel)`, or another provider changes that one line and nothing else.

## OpenRouter OAuth for BYOK

Expand Down Expand Up @@ -149,13 +149,14 @@ The log is append-only and ordered by `seq`. A consumer that reconnects after a

### `@inbrowser/model`

The shared `ModelClient` contract that relay and agent both consume, a set of cloud provider factories, and an on-device engine (Transformers.js + ONNX Runtime Web). Single root entrypoint — `import { … } from '@inbrowser/model'`.
The shared `ModelClient` contract that relay and agent both consume, cloud provider factories and adapters, and an on-device engine (Transformers.js + ONNX Runtime Web). Single root entrypoint — `import { … } from '@inbrowser/model'`.

**Cloud providers** — each is a factory that returns a `ModelClient` from `{ apiKey, model, … }`:
**Cloud providers** — each returns a `ModelClient`; most are `{ apiKey, model, … }` factories, while Firebase AI Logic wraps a model constructed by the host app:

| Factory | Config | Notes |
|---------|--------|-------|
| `geminiModelClient(config)` | `GeminiConfig` | Google AI Studio / Vertex |
| `createFirebaseAiLogicModelClient(model, opts?)` | Constructed Firebase `GenerativeModel` | Host owns Firebase, App Check, backend, and location; no Firebase dependency; not directly relay-registerable |
| `openrouterModelClient(config)` | `OpenRouterConfig` | Unified API, many models |
| `requestyModelClient(config)` | `RequestyConfig` | OpenAI-compatible gateway, many models |
| `anthropicModelClient(config)` | `AnthropicConfig` | Anthropic Claude |
Expand Down
10 changes: 6 additions & 4 deletions packages/agent/docs/how-to/implement-llm-client.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,12 @@ shared contract from `@inbrowser/model/contract`, re-exported from
provider and maps its stream to `ModelEvent`s. For the full event and usage
shapes, see the [`ModelClient` reference](../reference/library.md).

> Already have a provider? The cloud providers (Gemini, OpenRouter, Anthropic,
> Ollama, the Claude CLI/Code bridges) ship as `ModelClient` factories in
> `@inbrowser/model`. Import one and hand it to a session — you only need this
> guide when wiring an API the package does not cover.
> Already have a provider? The API-key and subscription providers (Gemini,
> OpenRouter, Anthropic, Ollama, and the Claude CLI/Code bridges) ship as
> `ModelClient` factories in `@inbrowser/model`. Firebase AI Logic instead uses
> `createFirebaseAiLogicModelClient` to wrap a caller-constructed Firebase model.
> Import one and hand it to a session — you only need this guide when wiring an
> API the package does not cover.

## Choose your path

Expand Down
14 changes: 8 additions & 6 deletions packages/agent/docs/reference/library.md
Original file line number Diff line number Diff line change
Expand Up @@ -319,12 +319,14 @@ interface ModelClient {
}
```

The narrow model interface. The cloud providers in `@inbrowser/model`
(`geminiModelClient`, `openrouterModelClient`, `anthropicModelClient`,
`ollamaModelClient`, `claudeCliModelClient`, `claudeCodeModelClient`) are
factories returning one. The client knows about model calls and streamed events;
it knows nothing about BYOK forms, storage, model pickers, or pricing. `chat`
yields `ModelEvent`s, enumerated in [events.md](./events.md). `id` is a stable
The narrow model interface. The API-key and subscription providers in
`@inbrowser/model` (`geminiModelClient`, `openrouterModelClient`,
`anthropicModelClient`, `ollamaModelClient`, `claudeCliModelClient`,
`claudeCodeModelClient`) are factories returning one.
`createFirebaseAiLogicModelClient` instead wraps a caller-constructed Firebase
AI Logic model. The client knows about model calls and streamed events; it knows
nothing about BYOK forms, storage, model pickers, or pricing. `chat` yields
`ModelEvent`s, enumerated in [events.md](./events.md). `id` is a stable
metrics/provenance string such as `gemini:gemini-3.5-flash`.

### `ModelRequest`
Expand Down
25 changes: 18 additions & 7 deletions packages/model/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,10 @@ The model layer. Two halves:
1. **Contract + cloud providers.** `src/contract.ts` defines the one
`ModelClient` contract the whole stack shares (relay + agent both
consume it). `src/providers/*` are the cloud providers (Gemini,
OpenRouter, Requesty, Anthropic, Ollama, Claude-CLI, Claude-Code), each a
factory returning a `ModelClient`. `src/with-retry.ts` decorates one.
Firebase AI Logic, OpenRouter, Requesty, Anthropic, Ollama, Claude-CLI,
Claude-Code), each returning a `ModelClient`. Firebase AI Logic is a
constructed-model adapter; the others are provider factories.
`src/with-retry.ts` decorates one.
2. **On-device engine.** Wraps `@huggingface/transformers` behind a
narrow `Engine` surface (`src/engine.ts`) that streams `EngineEvent`.

Expand All @@ -28,10 +30,12 @@ still-forthcoming piece — the adapter is the building block it needs.)
- `src/types.ts` is the canonical engine type surface. Engine-side files
import engine types from here.
- `src/engine.ts` is the only module that holds runtime model state.
- Each `src/providers/<name>.ts` is self-contained: it imports the
contract types and emits `ModelEvent`s. Providers do not import the
relay or the agent — the dependency points inward (relay/agent depend
on this package's contract, never the reverse).
- Each `src/providers/<name>.ts` imports the contract types and emits
`ModelEvent`s. Pure Gemini protocol helpers shared by the raw Gemini and
Firebase AI Logic transports live in `src/providers/gemini-protocol.ts`;
transport decoders remain provider-local. Providers do not import the relay
or the agent — the dependency points inward (relay/agent depend on this
package's contract, never the reverse).
- `src/worker.ts` returns the same `Engine` shape `createEngine`
returns. Consumers must not need to know which side of `postMessage`
the engine lives on.
Expand Down Expand Up @@ -65,6 +69,10 @@ Use the precise terms — they show up in types, comments, and PRs:
- Don't make `@huggingface/transformers` a regular dependency. It's
a peer dep; consumers control the version. (The Claude Code Agent SDK,
used only by `claudeCodeModelClient`, is an optional peer dep.)
- Don't make `firebase` a dependency or initialize Firebase/App Check in this
package. `createFirebaseAiLogicModelClient` accepts a structural,
caller-constructed `GenerativeModel`; the host owns its Firebase app,
backend, location, authentication, and App Check lifecycle.

## Status

Expand All @@ -73,4 +81,7 @@ consume a `ModelClient` from here. The engine loads and `generate()`
streams real tokens, and the engine is now a `ModelClient` via
`createEngineModelClient` (the engine→ModelClient adapter). The next
slice is the site wiring that drives a local engine through the agent
end to end (the in-browser docs-chat toggle).
end to end (the in-browser docs-chat toggle). Firebase AI Logic's core
text/thinking/custom-tool path is implemented through
`createFirebaseAiLogicModelClient`; its Live, Imagen, template, multimodal,
and hybrid lifecycle surfaces remain intentionally outside `ModelClient`.
32 changes: 31 additions & 1 deletion packages/model/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@ Two halves, one package:
providers (`geminiModelClient`, `openrouterModelClient`,
`requestyModelClient`, `anthropicModelClient`, `openaiCompatModelClient`,
`ollamaModelClient`, `llamaServerModelClient`, `claudeCliModelClient`,
`claudeCodeModelClient`) are factories that each return a `ModelClient`.
`claudeCodeModelClient`) and the Firebase AI Logic constructed-model
adapter (`createFirebaseAiLogicModelClient`) each return a `ModelClient`.
`withRetry` decorates one.
- **The on-device engine.** `createEngine` loads ONNX models in the
browser via `@huggingface/transformers` + ONNX Runtime Web (WebGPU /
Expand Down Expand Up @@ -58,6 +59,34 @@ The turn ends when the iterable returns; a `usage` event (or a terminal
`error` event) is the last thing emitted. There is no `turn_complete`
event.

## Firebase AI Logic from an existing Firebase app

Firebase AI Logic uses the Firebase app that the host already configured. The
host owns Firebase initialization, App Check, backend selection, Vertex AI
location, and construction of the `GenerativeModel`; this package only adapts
that model to `ModelClient`:

```ts
import { initializeApp } from 'firebase/app';
import { getAI, getGenerativeModel, GoogleAIBackend } from 'firebase/ai';
import { createFirebaseAiLogicModelClient } from '@inbrowser/model';

const app = initializeApp(firebaseConfig);
// Initialize App Check for `app` before making production AI requests.
const ai = getAI(app, { backend: new GoogleAIBackend() });
const firebaseModel = getGenerativeModel(ai, { model: 'gemini-3.5-flash' });
const client = createFirebaseAiLogicModelClient(firebaseModel);
```

The adapter streams text and thinking, translates caller-run custom function
calls (including thought-signature replay), maps sampling and usage, forwards
cancellation, and normalizes Firebase errors. It has no `firebase` dependency:
the constructed model crosses a small structural interface. Live API, Imagen,
server templates, Firebase built-in/automatic tools, multimodal events,
structured-output configuration, token counting, and hybrid lifecycle control
are intentionally outside this adapter. See the
[Firebase AI Logic reference](docs/reference/firebase-ai-logic.md).

## A local OpenAI-compatible server

Ollama, llama.cpp's `llama-server`, vLLM, LM Studio, LocalAI, and friends all
Expand Down Expand Up @@ -134,6 +163,7 @@ Everything is imported from the package root `@inbrowser/model`.
|---|---|
| `ModelClient`, `ModelRequest`, `ModelEvent`, `ModelMessage`, `ModelUsage`, `ToolSpec`, `ReasoningEffort` | The shared contract (type-only) |
| `geminiModelClient`, `openrouterModelClient`, `requestyModelClient`, `anthropicModelClient`, `openaiCompatModelClient`, `ollamaModelClient`, `llamaServerModelClient`, `claudeCliModelClient`, `claudeCodeModelClient` | Cloud + local provider factories; each returns a `ModelClient` |
| `createFirebaseAiLogicModelClient(model, opts?)` | Wraps a caller-constructed Firebase AI Logic `GenerativeModel`; Firebase/App Check lifecycle stays with the host |
| `OpenAiCompatConfig`, `OllamaConfig`, `LlamaServerConfig` | Config shapes for the OpenAI-compatible factory and its local presets |
| `withRetry(client, opts?)` | Decorator that retries transient upstream errors while nothing has streamed |
| `CloudProviderConfig`, `ModelClientFactory` | Shared provider config + the factory type the relay routes on |
Expand Down
5 changes: 4 additions & 1 deletion packages/model/docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@
`ModelClient` contract (from `@inbrowser/model`) that both `@inbrowser/relay`
(transport) and `@inbrowser/agent` (runtime) consume, the cloud providers that
implement it (Gemini, OpenRouter, Requesty, Anthropic, Ollama, Claude-CLI,
Claude-Code), and the on-device LLM engine. The engine loads ONNX models in the browser
Claude-Code, plus the Firebase AI Logic constructed-model adapter), and the
on-device LLM engine. The engine loads ONNX models in the browser
through `@huggingface/transformers` (ONNX Runtime Web over WebGPU or WASM) and
exposes them behind a narrow `Engine` surface that streams `EngineEvent`s; a
worker transport lets the same engine run off the main thread without any
Expand Down Expand Up @@ -48,10 +49,12 @@ The facts: configuration shapes, event variants, exports.
- [Presets](reference/presets.md)
- [Worker](reference/adapters-and-worker.md)
- [Gateway providers](reference/gateway-providers.md)
- [Firebase AI Logic](reference/firebase-ai-logic.md)

## Explanation

Background and design rationale.

- [Design](explanation/design.md)
- [On-device inference](explanation/on-device-inference.md)
- [Firebase AI Logic provider assessment](explanation/firebase-ai-logic-provider-assessment.md)
1 change: 1 addition & 0 deletions packages/model/docs/reference/engine.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ Everything is imported from the package root `@inbrowser/model`.
| `createEngine`, `definePreset`, `parseToolCalls`, `splitThinking`, and engine types | The on-device engine surface |
| The six bundled presets | `deepseek_r1_qwen_1_5b`, `gemma4_E2B`, `gemma4_E4B`, `qwen2_5_coder_1_5b`, `qwen3_1_7b`, `smollm2_360m`. See [./presets.md](./presets.md). |
| The cloud provider factories (`geminiModelClient`, …), `withRetry` | Cloud providers + the retry decorator |
| `createFirebaseAiLogicModelClient` | Wraps a caller-constructed Firebase AI Logic model as a `ModelClient`; it is not an API-key cloud factory. |
| `ModelClient`, `ModelRequest`, `ModelEvent`, `ModelMessage`, `ModelUsage`, `ToolSpec`, `ReasoningEffort` | The shared `ModelClient` contract types (type-only) |
| `createEngineModelClient` | Wraps an `Engine` as a `ModelClient`. See [./adapters-and-worker.md](./adapters-and-worker.md). |
| `hostEngineInWorker`, `connectWorkerEngine` | Worker host/connect helpers. See [./adapters-and-worker.md](./adapters-and-worker.md). |
Expand Down
126 changes: 126 additions & 0 deletions packages/model/docs/reference/firebase-ai-logic.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
# Firebase AI Logic

`createFirebaseAiLogicModelClient()` wraps a caller-constructed Firebase AI
Logic `GenerativeModel` as the package's shared `ModelClient`.

## Construct the Model in the Host

The application owns Firebase initialization, Authentication, App Check,
Gemini backend selection, Vertex AI location, and model construction:

```ts
import { initializeApp } from 'firebase/app';
import { getAI, getGenerativeModel, GoogleAIBackend } from 'firebase/ai';
import { createFirebaseAiLogicModelClient } from '@inbrowser/model';

const app = initializeApp(firebaseConfig);
// Configure App Check for this app before production requests.
const ai = getAI(app, { backend: new GoogleAIBackend() });
const model = getGenerativeModel(ai, { model: 'gemini-3.5-flash' });

const client = createFirebaseAiLogicModelClient(model);
```

To use the Vertex AI Gemini API, construct the `AI` instance with the
appropriate `VertexAIBackend` and location before creating the model. The
adapter does not inspect or change that choice.

`@inbrowser/model` does not depend on `firebase`. The adapter accepts the
narrow structural shape it uses, so the host installs Firebase and controls
its version and lifecycle.

## API

```ts
createFirebaseAiLogicModelClient(model, options?): ModelClient
```

`model` must provide:

```ts
interface FirebaseAiLogicGenerativeModelLike {
readonly model: string;
generateContentStream(
request: unknown,
options?: { signal?: AbortSignal },
): Promise<{
stream: AsyncIterable<unknown>;
response: Promise<unknown>;
}>;
}
```

Options:

| Option | Meaning |
| --- | --- |
| `id` | Metrics/provenance id; defaults to `firebase-ai-logic:${model.model}` |
| `temperature` | Construction-time default; a `ModelRequest.temperature` value wins |

The returned client has `supportsTools: true`.

## Supported Mapping

Each `chat()` call is stateless and sends the full `ModelRequest` through
`generateContentStream()`; it does not create a Firebase `ChatSession`.

- System messages are joined into one Firebase system instruction.
- User, assistant, and adjacent tool-result messages become Firebase `user`,
`model`, and grouped `function` content.
- Custom `ToolSpec` declarations are converted to Firebase function
declarations. Common JSON Schema-only annotations are removed; unsupported
structural or validation keywords fail as a terminal, non-retryable error
instead of being sent ambiguously.
- Tool execution remains with `@inbrowser/agent` or the caller. Firebase
`functionReference` and automatic tool execution are not used.
- Upstream function-call ids are preserved. Calls without one receive a stable
local id; local ids are not replayed to Firebase.
- Gemini thought signatures are captured from function-call parts and replayed
on the next request.
- Temperature, top-p, top-k, and `ReasoningEffort` are mapped. Gemini 3.x uses
uppercase thinking levels; Gemini 2.5 uses thinking budgets. `off` leaves the
model default unchanged.
- Requests use a `65,536` maximum output-token budget so large function
arguments are not silently truncated.

The event stream contains:

- `text` for visible text;
- `thinking` for returned thought summaries;
- completed `tool_call` events after streamed call snapshots are assembled;
- one terminal `usage` event using the latest prompt, output, cached, and
thinking-token counts;
- or one terminal `error` event with normalized Firebase status/details,
including prompt and candidate-level content blocks.

The caller's `AbortSignal` is forwarded to Firebase. Caller cancellation ends
the stream silently, matching the other `ModelClient` implementations.

## Deliberate Exclusions

This adapter covers the common text and caller-run custom-tool path. It does
not expose:

- Gemini Live API sessions;
- Imagen;
- server-side prompt templates;
- Firebase built-in Search, Maps, URL-context, or code-execution tools;
- Firebase automatic function execution;
- multimodal input/output events;
- structured-output or safety-setting request fields;
- token counting or model discovery;
- Firebase hybrid/on-device lifecycle controls.

Those capabilities need contracts other than the current text/tool-oriented
`ModelClient` rather than more switches on this adapter.

## Relay Compatibility

The adapter is not directly a `ModelClientFactory`. The relay factory receives
only `{ apiKey?, model }`, while Firebase AI Logic requires a constructed model
bound to a Firebase app and security context. The relay also currently requires
a BYOK or server-managed key before invoking a provider. Page-direct use works;
first-class relay routing needs a separate provider-authentication policy.

For the design and feature ROI, see the
[Firebase AI Logic provider assessment](../explanation/firebase-ai-logic-provider-assessment.md).
2 changes: 1 addition & 1 deletion packages/model/package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "@inbrowser/model",
"version": "0.4.1",
"description": "The model layer for the inbrowser stack: it OWNS the shared ModelClient contract that @inbrowser/relay (transport) and @inbrowser/agent (runtime) both consume, the cloud provider factories (Gemini, OpenRouter, Requesty, Anthropic, Ollama, Claude-CLI, Claude-Code) that each return a ModelClient, a withRetry decorator, and the on-device LLM engine (lazy-loads @huggingface/transformers + ONNX Runtime Web behind a narrow EngineEvent-streaming Engine surface; Gemma 4 + Qwen + SmolLM2 presets; in-worker host/connect helpers). The engine is also a ModelClient via createEngineModelClient. Everything is exported from the package root.",
"description": "The model layer for the inbrowser stack: it OWNS the shared ModelClient contract that @inbrowser/relay (transport) and @inbrowser/agent (runtime) both consume, cloud provider factories, a constructed-model Firebase AI Logic adapter, a withRetry decorator, and the on-device LLM engine (lazy-loads @huggingface/transformers + ONNX Runtime Web behind a narrow EngineEvent-streaming Engine surface; Gemma 4 + Qwen + SmolLM2 presets; in-worker host/connect helpers). The engine is also a ModelClient via createEngineModelClient. Everything is exported from the package root.",
"type": "module",
"main": "./dist/index.js",
"types": "./dist/index.d.ts",
Expand Down
Loading
Loading