Local no-API-key mode: Ollama sidecar + compact local runtime

[srkprasad1995]

What

Run the whole stack with zero API keys: uncomment COMPOSE_PROFILES=local in .env and make up. An Ollama sidecar serves a small local text model + code embeddings, with both models baked into the image at build time (~2 GB download at first build; nothing downloads after the stack is up). Real keys, when present, always win — a keyed setup is byte-identical to today.

How

Engine — LocalOllamaRuntime (engine_core/local_runtime.py)

• With no key configured, the runner dispatches jobs to a compact tool-calling loop over Ollama's native /api/chat via the new AgentRuntime.local_runtime() hook — not the Claude CLI, whose ~21.5K-token scaffolding takes minutes of CPU prompt-eval per turn.
• Exposes the workspace tools (Read/Glob/Grep) + the attached MCP servers' tools (CLAUDE_ENGINE_LOCAL_REMOTE_TOOLS, default * = all; narrow it if a small model stops calling tools).
• Own FS session store (the orchestrator sends only the new message + sessionId); streams the standard protocol events.
• Memory dream jobs inherit local mode automatically (same path).

Indexer — auto embedding provider

• Default flips voyage → auto: voyage (key) → openai (key) → ollama (sidecar) — no key now means local embeddings instead of "semantic search disabled".
• New OllamaProvider over /api/embed (1024-dim qwen3-embedding:0.6b, matches DEXIASK_EMBEDDING_DIM; retry/backoff, offline constructor, dim guard).

Sidecar — ollama/

• Models pulled at build into plain image layers (OLLAMA_MODELS=/models; /root/.ollama is a shadowed VOLUME).
• Tuning baked into the text model's Modelfile only (embedding model keeps its small KV cache): num_ctx 16384, num_thread 4 — pinned threads measured 31.6 vs 0.1 tok/s against llama.cpp's all-cores default on laptops.
• Entrypoint pre-loads the text model so the first turn skips cold load; OLLAMA_NUM_PARALLEL=1 so the prefix cache makes follow-up turns fast.
• Default model qwen2.5:1.5b — deliberately non-reasoning (reasoning models burn the whole turn "thinking" on CPU) and tool-capable.

Verified end-to-end (no keys, CPU-only laptop)

• Chat turn: answered in ~27s, streamed; follow-up turn ~5s (session resume + prefix cache)
• Codebase question: model called semantic_search → grounded answer naming the right file, ~50s
• Index + semantic search over local embeddings ✅
• Keys-win regression: profile off renders byte-identical compose config; job/env keys bypass the local path (unit-tested)

Tests

• Engine: 139 passed (new tests/core/test_local_runtime.py — streaming, tool dispatch, session resume, truncation, arg parsing; runner swap/precedence tests), ruff clean
• Indexer: 182 passed (OllamaProvider + auto resolution matrix), ruff clean
• Backend/memory go test + go vet, web vitest: green

🤖 Generated with Claude Code

https://claude.ai/code/session_01D6Wso8dGjAk7QrjvHWy5ma