[Extensibility 3/4] feat: local LLMs via Ollama, behind ENABLE_OLLAMA (stacked on #259) - #261
[Extensibility 3/4] feat: local LLMs via Ollama, behind ENABLE_OLLAMA (stacked on #259)#261amal66 wants to merge 2 commits into
Conversation
|
Keeping this in draft — an adversarial re-review found three confirmed bugs to fix first: (1) the https-in-production check fires before |
Replace the per-provider if/else chains in lib/llm/index.ts and the env-var switch in lib/userApiKeys.ts with a provider registry (lib/llm/registry.ts) and an API-key provider table (core/apiKeyProviders.ts). Adding a provider is now a registerProvider()/registerApiKeyProvider() call — no edits to index.ts, models.ts, or userApiKeys.ts required. Ported from the amal66/mike monorepo fork (origin/main, b3166dd); mechanical translation into the backend/ layout. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CEguyEgXa9JjCciXCcVemC
…OLLAMA) Mechanical port from b3166dd (apps/api -> backend): - lib/llm/providers/ollama.ts: Ollama adapter registered through the provider registry, gated by ENABLE_OLLAMA (off by default) - lib/llm/baseUrl.ts: OPENAI_BASE_URL resolution with SSRF guard (OPENAI_ALLOW_LOCAL_BASE_URL) so the OpenAI adapter can point at a local Ollama server - lib/privateIp.ts: shared private/reserved IP classifier used by the base-URL guard - lib/llm/openai.ts: fetch openAIResponsesUrl() instead of the hard-coded api.openai.com constant - lib/llm/index.ts: register Ollama in registerBuiltinProviders() when ENABLE_OLLAMA=true - .env.example: document the opt-in Ollama configuration - tests for the ollama gate, base-URL resolution, and IP classifier Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CEguyEgXa9JjCciXCcVemC
|
Closing as superseded by the Ollama implementation already on The current implementation includes dynamic model discovery, frontend model selection, tool-calling with conversation context preserved, configurable connection settings, optional authentication, and documentation. This draft is also conflicting and its known issues are addressed by the implementation now on |
2 participants
Closes #23.
Summary
Opt-in local LLM support via Ollama's OpenAI-compatible endpoint: a self-hosted deployment can run chat inference entirely on its own hardware — documents never leave the building, and with this flag neither does inference. Everything is behind
ENABLE_OLLAMA=true(default off): with the flag unset, no new models appear and nothing changes.Changes
providers/ollama.ts(new): registry entry that routes local model IDs (llama3.3, phi4, qwen2.5, +OLLAMA_MODELSextras) through the existing OpenAI adapter pointed athttp://localhost:11434/v1. No new dependencies, no new streaming code path.baseUrl.ts(new) +openai.ts:OPENAI_BASE_URLresolution with an SSRF guard — http/localhost/private-IP endpoints rejected in production unless explicitly allowed. Defaults tohttps://api.openai.com/v1(unchanged when unset).privateIp.ts(new): shared private/reserved IP classifier (IPv4 + IPv6 incl. mapped/NAT64/6to4)..env.example: documents the opt-in config.Why
The most-requested self-hosting capability (#23), done as a registry entry instead of a bespoke client — the first external provider to use the registry, which is exactly what it was built for.
Testing
On this branch: backend
npm test— 300 passed, 5 skipped (includes the 21 new tests). Flag gate verified: only the exact string"true"registers the provider.Provenance
Mechanical port of code running in amal66/mike (main);
privateIp.tsandproviders/ollama.tsbyte-identical to the fork. Full provenance in amal66#34.🤖 Generated with Claude Code