Local no-API-key mode: Ollama sidecar + compact local runtime

.env.example

@@ -1,10 +1,41 @@
 # ============================================================================
 # Dexiask environment — copy to `.env` and fill in the secrets.
 #   cp .env.example .env
-# All secrets are brought by you; nothing is baked into images.
+# All secrets are brought by you. No API keys at all? Enable local mode below.
 # ============================================================================
 
-# --- Secrets (required) --------------------------------------------------
+# --- Local mode (run with NO API keys) -------------------------------------
+# Uncomment to add an Ollama sidecar that serves a small local text model and
+# local code embeddings. Both models are baked into the sidecar image at build
+# time (first build downloads ~2 GB; nothing is downloaded after the stack is
+# up). With no keys set, the engine and indexer fall back to it automatically;
+# real keys, when present, always win. Runs CPU-only on a regular laptop
+# (~3 GB RAM); expect simpler answers than Claude. Switching between local and
+# hosted embeddings requires a re-index (`make clean` or per-repo reindex):
+# same dimension, different vector space.
+#COMPOSE_PROFILES=local
+# Models baked into the sidecar image (build args). Pick a non-reasoning,
+# tool-capable text model; bigger = better answers but more RAM/slower on CPU
+# (e.g. qwen2.5:7b ~4.7 GB). The embedding model must produce
+# DEXIASK_EMBEDDING_DIM-sized vectors (qwen3-embedding:0.6b → 1024).
+#DEXIASK_LOCAL_TEXT_MODEL=qwen2.5:1.5b
+#DEXIASK_LOCAL_EMBED_MODEL=qwen3-embedding:0.6b
+# Tuning baked into the text model at build time (build args — rebuild to
+# apply). Context: Ollama's 4096 default would truncate the agent's system
+# prompt; larger = more RAM. Threads: llama.cpp defaults to every core, which
+# on laptops is dramatically SLOWER than a few pinned threads; raise only on
+# real many-core servers.
+#DEXIASK_OLLAMA_CONTEXT_LENGTH=16384
+#DEXIASK_OLLAMA_THREADS=4
+# Remote MCP tools exposed to the local model: '*' (default) = every tool the
+# indexer/memory servers offer. If a small model stops calling tools and
+# answers from thin air, narrow this to a short list — fewer tools are easier
+# for small models to follow:
+#DEXIASK_LOCAL_REMOTE_TOOLS=semantic_search,get_chunk,read_range,get_overview,memory_search
+# On a CPU-only box, consider disabling the hourly dream consolidation:
+#DEXIASK_DREAM_INTERVAL=0
+
+# --- Secrets (optional in local mode, required otherwise) ------------------
 # Anthropic API key used by the Claude engine. This may also be a key for an
 # Anthropic-compatible gateway (see ANTHROPIC_BASE_URL below).
 ANTHROPIC_API_KEY=
@@ -16,6 +47,9 @@ ANTHROPIC_API_KEY=
 ANTHROPIC_BASE_URL=
 # Voyage AI key used by the indexer for code embeddings. Get one at voyageai.com.
 VOYAGE_API_KEY=
+# Optional alternative embedding key; the indexer picks the first configured
+# provider: Voyage → OpenAI → local Ollama sidecar.
+OPENAI_API_KEY=
 
 # --- Slack bot (optional) ------------------------------------------------
 # Leave blank to disable the Slack bot. Both must be set to enable it.

CLAUDE.md

@@ -139,10 +139,25 @@ manages them via the `/api/mcp/*` BFF proxy.
 ## Environment
 
 Copy `.env.example` to `.env`. All secrets come from env (`ANTHROPIC_API_KEY`,
-`VOYAGE_API_KEY`, `SLACK_*`); nothing is baked into images. Service URLs use
+`VOYAGE_API_KEY`, `SLACK_*`); no secrets are baked into images. Service URLs use
 compose-internal DNS (`http://engine:8080`, `http://indexer:8080`, `http://memory:8080`,
 `http://qdrant:6333`, `postgres:5432`).
 
+**Local mode (no API keys)**: `COMPOSE_PROFILES=local` in `.env` adds an `ollama`
+sidecar whose text + embedding models (`DEXIASK_LOCAL_TEXT_MODEL` /
+`DEXIASK_LOCAL_EMBED_MODEL`, defaults `qwen2.5:1.5b` / `qwen3-embedding:0.6b`) are
+deliberately **baked into the image at build time** — nothing downloads after the
+stack is up. The profile injects `CLAUDE_ENGINE_LOCAL_BASE_URL` /
+`DEXIASK_OLLAMA_BASE_URL` (= `http://ollama:11434`); with no API key the engine
+dispatches jobs to `engine_core.local_runtime.LocalOllamaRuntime` — a compact
+tool-calling loop over Ollama's native `/api/chat` (role prompt + workspace tools +
+the `local_remote_tools` MCP allowlist; no Claude CLI, whose scaffolding is far too
+many prompt tokens for a small CPU model) with its own FS session store — overriding
+the Job's model with the local one (dream jobs included). The indexer's `auto`
+embedding provider resolves voyage → openai → ollama. Real keys always win. Local
+and hosted embeddings share a dim (1024) but not a vector space: switching requires
+a re-index.
+
 Feature-gating env (all optional; each degrades gracefully when unset):
 - **Auth**: `DEXIASK_GITHUB_CLIENT_ID`/`_SECRET` (empty → dev-fallback, no login),
   `DEXIASK_OAUTH_CALLBACK_URL`, `DEXIASK_WEB_BASE_URL`, `DEXIASK_SESSION_SECRET`,

Makefile

@@ -2,7 +2,7 @@
 .DEFAULT_GOAL := help
 COMPOSE := docker compose
 
-.PHONY: help up down build logs ps restart test test-backend test-memory test-engine test-indexer test-web lint fmt clean
+.PHONY: help up up-local down build logs ps restart test test-backend test-memory test-engine test-indexer test-web lint fmt clean
 
 help: ## Show this help
 	@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | \
@@ -12,6 +12,10 @@ up: ## Build (if needed) and start the whole stack
 	$(COMPOSE) up --build -d
 	@echo "web → http://localhost:$${DEXIASK_WEB_PORT:-25051}"
 
+up-local: ## Start with the local no-API-key Ollama sidecar (same as COMPOSE_PROFILES=local in .env)
+	COMPOSE_PROFILES=local $(COMPOSE) up --build -d
+	@echo "web → http://localhost:$${DEXIASK_WEB_PORT:-25051}"
+
 down: ## Stop the stack
 	$(COMPOSE) down
 

README.md

@@ -101,14 +101,17 @@ it directly via semantic + lexical search.
 - **memory** — Go. FS-backed user/repo/global memory exposed as an MCP server, with a
   periodic "dream" consolidation judge. No database.
 - **qdrant / postgres** — vector store and relational store.
+- **ollama** *(optional, `COMPOSE_PROFILES=local`)* — local text + embedding models
+  baked into the image, so everything runs with no API keys.
 
 One **shared `/workspace` mount** is the codebase the agent reads and the indexer
 indexes (read-only for the indexer).
 
 ## Quickstart
 
 **Prerequisites:** Docker + Docker Compose, an [Anthropic API key](https://console.anthropic.com/),
-and a [Voyage AI key](https://www.voyageai.com/) (for code embeddings).
+and a [Voyage AI key](https://www.voyageai.com/) (for code embeddings) — or **no keys
+at all** with [local mode](#local-mode-no-api-keys) below.
 
 ```bash
 git clone <your-fork> dexiask && cd dexiask
@@ -121,6 +124,48 @@ make up          # or: docker compose up --build -d
 Open **http://localhost:25051** and start chatting. `make logs` to tail, `make down`
 to stop, `make clean` to wipe the DB + index.
 
+### Local mode (no API keys)
+
+No Anthropic or Voyage key? Enable the **Ollama sidecar** and run entirely locally:
+
+```bash
+cp .env.example .env
+#  → edit .env: uncomment COMPOSE_PROFILES=local (leave the key vars blank)
+make up
+```
+
+The sidecar serves a small local text model (`qwen2.5:1.5b`) and local code
+embeddings (`qwen3-embedding:0.6b`). Both models are **baked into the sidecar
+image at build time** — the first build downloads ~2 GB; nothing is downloaded
+after the stack is up. With no keys configured the engine and indexer fall back
+to the sidecar automatically (the engine switches to a compact local agent loop
+sized for small models — first answer in ~30s on a laptop CPU, follow-ups in
+seconds); if you later add real keys they take over (hosted keys always win).
+
+Worth knowing:
+
+- **Quality/speed**: a small local model is noticeably less capable than Claude —
+  fine for exploring a codebase, not a Claude replacement. For better answers at
+  a higher resource cost, set a bigger **non-reasoning** model, e.g.
+  `DEXIASK_LOCAL_TEXT_MODEL=qwen2.5:7b` (~4.7 GB). Avoid reasoning models
+  (qwen3, deepseek-r1): they burn the whole turn "thinking" on CPU.
+- **Resources**: runs CPU-only on a regular laptop — ~4 GB free disk for the
+  image and ~3 GB RAM while running. The sidecar pins generation to 4 threads
+  (`DEXIASK_OLLAMA_THREADS`) — counter-intuitively much faster than all cores on
+  laptops. On an NVIDIA GPU host, uncomment the GPU block on the `ollama`
+  service in `docker-compose.yml` for faster responses.
+- **Tools**: the local agent gets every indexer/memory MCP tool by default
+  (`DEXIASK_LOCAL_REMOTE_TOOLS=*`). If a small model stops calling tools and
+  answers from thin air, narrow it to a short list (e.g.
+  `semantic_search,get_chunk,read_range`) — fewer tools are easier to follow.
+- **CPU-only boxes**: set `DEXIASK_DREAM_INTERVAL=0` to skip the hourly memory
+  consolidation model runs.
+- **Re-index on switch**: local and hosted embeddings share a dimension but not a
+  vector space — switching providers requires `make clean` (or a per-repo reindex).
+- Models are overridable via `DEXIASK_LOCAL_TEXT_MODEL` / `DEXIASK_LOCAL_EMBED_MODEL`
+  (image build args; the embedding model must produce `DEXIASK_EMBEDDING_DIM`-sized
+  vectors).
+
 ## Indexing a repository
 
 The indexer indexes the **default branch** of any git repo under your mounted
@@ -214,8 +259,10 @@ Everything is env-driven — see `.env.example` for the full list. The essential
 
 | Variable | Purpose |
 |---|---|
-| `ANTHROPIC_API_KEY` | Claude engine credential (required) |
-| `VOYAGE_API_KEY` | Indexer embedding credential (required) |
+| `COMPOSE_PROFILES=local` | Enable the Ollama sidecar — run with **no API keys** (see [Local mode](#local-mode-no-api-keys)) |
+| `ANTHROPIC_API_KEY` | Claude engine credential (required unless local mode) |
+| `VOYAGE_API_KEY` | Indexer embedding credential (required unless local mode) |
+| `DEXIASK_LOCAL_TEXT_MODEL` / `_LOCAL_EMBED_MODEL` | Models baked into the local sidecar image (defaults `qwen2.5:1.5b` / `qwen3-embedding:0.6b`) |
 | `DEXIASK_MODEL` | Claude model for ask mode (default `claude-sonnet-5`) |
 | `DEXIASK_WORKSPACE_PATH` | Host codebase mounted at `/workspace` |
 | `DEXIASK_SESSION_SECRET` / `_TOKEN_ENC_KEY` | Session signing + GitHub-token encryption — enables auth (token login) |

docker-compose.yml

@@ -35,15 +35,62 @@ services:
       timeout: 5s
       retries: 10
 
+  # Local no-API-key mode: `COMPOSE_PROFILES=local` in .env enables this sidecar
+  # and the fallback env below. Models are baked into the image at build time
+  # (build args) — nothing is downloaded after the stack is up.
+  ollama:
+    build:
+      context: ./ollama
+      args:
+        TEXT_MODEL: ${DEXIASK_LOCAL_TEXT_MODEL:-qwen2.5:1.5b}
+        EMBED_MODEL: ${DEXIASK_LOCAL_EMBED_MODEL:-qwen3-embedding:0.6b}
+        # Tuning baked into the TEXT model only (not global env), so the tiny
+        # embedding model keeps its small default context. See ollama/Dockerfile
+        # for why (KV-cache size; thread-contention slowdown on laptops).
+        TEXT_CONTEXT: ${DEXIASK_OLLAMA_CONTEXT_LENGTH:-16384}
+        TEXT_THREADS: ${DEXIASK_OLLAMA_THREADS:-4}
+    profiles: ["local"]
+    environment:
+      OLLAMA_MODELS: /models
+      # Keep models resident (text + embed loaded side by side).
+      OLLAMA_KEEP_ALIVE: -1
+      OLLAMA_MAX_LOADED_MODELS: 2
+      # One slot → the llama.cpp prefix cache holds the conversation prefix
+      # between turns, so turn N only evaluates the delta.
+      OLLAMA_NUM_PARALLEL: 1
+    healthcheck:
+      test: ["CMD", "ollama", "list"]
+      interval: 10s
+      timeout: 5s
+      retries: 10
+    # On an NVIDIA GPU host, uncomment to offload inference:
+    # deploy:
+    #   resources:
+    #     reservations:
+    #       devices:
+    #         - driver: nvidia
+    #           count: all
+    #           capabilities: [gpu]
+
   engine:
     build:
       context: ./engine
     environment:
-      ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY:?set ANTHROPIC_API_KEY in .env}
+      # Optional when the `local` profile is enabled — the engine then falls
+      # back to the Ollama sidecar. Required for Claude-quality answers.
+      ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY:-}
       # Optional: point the Claude SDK at an Anthropic-compatible gateway
       # (e.g. OpenRouter → https://openrouter.ai/api). Blank = api.anthropic.com.
       ANTHROPIC_BASE_URL: ${ANTHROPIC_BASE_URL:-}
       CLAUDE_ENGINE_MODEL: ${DEXIASK_MODEL:-claude-sonnet-5}
+      # Local fallback (set only when the `local` profile is on): with no API
+      # key the engine runs jobs against the Ollama sidecar's Anthropic-compat
+      # API using this model instead of erroring. A real key always wins.
+      CLAUDE_ENGINE_LOCAL_BASE_URL: ${COMPOSE_PROFILES:+http://ollama:11434}
+      CLAUDE_ENGINE_LOCAL_MODEL: ${DEXIASK_LOCAL_TEXT_MODEL:-qwen2.5:1.5b}
+      # Remote MCP tools exposed to the local model: '*' = all, or a comma
+      # allowlist — small models often follow tools better with fewer of them.
+      CLAUDE_ENGINE_LOCAL_REMOTE_TOOLS: ${DEXIASK_LOCAL_REMOTE_TOOLS:-*}
       # No OTel collector ships with the stack; disable the exporter to keep logs
       # clean. Set to "false" and add a collector to enable tracing.
       OTEL_SDK_DISABLED: ${OTEL_SDK_DISABLED:-true}
@@ -62,10 +109,16 @@ services:
     build:
       context: ./indexer
     environment:
-      DEXIASK_VOYAGE_API_KEY: ${VOYAGE_API_KEY:?set VOYAGE_API_KEY in .env}
+      # Optional when the `local` profile is enabled — the `auto` provider then
+      # falls back to the Ollama sidecar for embeddings. A real key always wins.
+      DEXIASK_VOYAGE_API_KEY: ${VOYAGE_API_KEY:-}
       DEXIASK_QDRANT_URL: ${DEXIASK_QDRANT_URL:-http://qdrant:6333}
+      DEXIASK_EMBEDDING_PROVIDER: ${DEXIASK_EMBEDDING_PROVIDER:-auto}
       DEXIASK_EMBEDDING_MODEL: ${DEXIASK_EMBEDDING_MODEL:-voyage-code-3}
       DEXIASK_EMBEDDING_DIM: ${DEXIASK_EMBEDDING_DIM:-1024}
+      DEXIASK_OPENAI_API_KEY: ${OPENAI_API_KEY:-}
+      DEXIASK_OLLAMA_BASE_URL: ${COMPOSE_PROFILES:+http://ollama:11434}
+      DEXIASK_OLLAMA_EMBEDDING_MODEL: ${DEXIASK_LOCAL_EMBED_MODEL:-qwen3-embedding:0.6b}
       # Generated domain-knowledge docs (LLM), embedded as content_type=doc and
       # searchable alongside code. Off by default; needs an Anthropic key.
       DEXIASK_ENABLE_DOMAIN_DOCS: ${DEXIASK_ENABLE_DOMAIN_DOCS:-false}

engine/.env.example

@@ -11,6 +11,16 @@ CLAUDE_ENGINE_MODEL=claude-sonnet-5
 # Anthropic-compatible gateway base URL (e.g. a local proxy). Empty = default.
 ANTHROPIC_BASE_URL=
 
+# ── Local model fallback (no API key) ───────────────────────────────────────
+# With no key configured anywhere, jobs run on the compact local runtime against
+# this Ollama server (the compose `local` profile's sidecar) with the local
+# model replacing the Job's. Empty = fallback unavailable (missing key errors).
+# CLAUDE_ENGINE_LOCAL_BASE_URL=http://ollama:11434
+# CLAUDE_ENGINE_LOCAL_MODEL=qwen2.5:1.5b
+# Remote MCP tools the local runtime exposes (small models need few tools):
+# CLAUDE_ENGINE_LOCAL_REMOTE_TOOLS=*   # or a comma allowlist, e.g. semantic_search,get_chunk
+# CLAUDE_ENGINE_LOCAL_REQUEST_TIMEOUT_S=300
+
 # HTTP server bind (container listens on 8080).
 CLAUDE_ENGINE_SERVER_HOST=0.0.0.0
 CLAUDE_ENGINE_SERVER_PORT=8080

engine/engine_core/capabilities/mcp_router.py

@@ -89,6 +89,14 @@ async def _connect(self, name: str) -> AsyncIterator[ClientSession]:
                     await session.initialize()
                     yield session
 
+    async def catalog(self, name: str) -> list[dict[str, Any]]:
+        """The cached tool list for *name* (fetched on first use).
+
+        Public for consumers that need exact tool names/schemas rather than a
+        capability search (the local runtime's tool allowlist).
+        """
+        return await self._ensure_catalog(name)
+
     async def _ensure_catalog(self, name: str) -> list[dict[str, Any]]:
         """Fetch + cache the tool list for *name* (once per run)."""
         if name in self._catalog: