Skip to content

ARCHITECTURE.md

Latest commit

 

History

History
317 lines (245 loc) · 13.1 KB

ARCHITECTURE.md

File metadata and controls

317 lines (245 loc) · 13.1 KB
Raw

CLI-JAW Architecture

Technical overview for contributors. The detailed source-of-truth map lives in structure/; start with structure/INDEX.md, then use structure/str_func.md, structure/server_api.md, and structure/commands.md for exact counts and route/command surfaces.

This file is intentionally higher level than structure/ so it can stay useful in public and submodule-light checkouts.

The static HTML site under docs/dev/ is a browsable mirror; when it diverges, treat structure/ plus the Markdown files in docs/ as source of truth.

System Overview

User interfaces
  Web chat UI          public/
  Terminal TUI        bin/commands/chat.ts + src/cli/tui/
  Telegram            src/telegram/
  Discord             src/discord/
  Electron desktop    electron/ + src/manager/

Core server
  server.ts           bootstrap, auth, security, WS/SSE, route mounting
  src/routes/         extracted REST/SSE route modules
  src/core/           config, db, event bus, settings, employees
  src/http/           ok/fail response helpers and error middleware
  src/security/       path, filename, origin, and audit guards

Agent runtime
  src/agent/          spawn, resume, lifecycle, retry, watchdog
  src/agent/events/   provider event adapters and tool-label mapping
  src/cli/registry.ts 13 runtime registry entries and defaults
  src/prompt/         prompt assembly, skills, runtime context, memory injection

Workflow and memory
  src/orchestrator/   IPABCD/PABCD state machine, dispatch, worker progress
  src/goal/           persistent goal lifecycle and completion evidence gates
  src/goal-run/       bounded goal-run preview state
  src/task/           agent-native task checklist store
  src/team/           multi-agent dispatch planning
  src/jaw-ceo/        OpenAI Realtime CEO channel
  src/memory/         local structured memory, heartbeat schedules, indexing

Manager dashboard
  src/manager/        multi-instance dashboard server, notes, board, reminders,
                      schedule, connector, git diff, memory federation
  public/manager/     React manager frontend

Browser and automation
  src/browser/        CDP primitives, runtime diagnostics, tab lifecycle
  src/browser/web-ai/ ChatGPT/Gemini/Grok web-AI session automation (96 TS files)
  src/browser/adaptive-fetch/ URL reader + browser-escalation pipeline (34 files;
                      typed stage scheduler, SSRF guards, warm pool, BM25, Camoufox/yt-dlp/Jina)

Current core API shape, as of June 27, 2026 (see structure/INDEX.md footer and structure/server_api.md / structure/commands.md for live counts):

  • server.ts: 635 lines of glue/bootstrap (auth, security, SSE bootstrap, route mounting).
  • REST/SSE routes: 232 handlers including / across server.ts, src/routes/*.ts, and mounted sub-routers.
  • Browser API: 43 handlers in src/routes/browser.ts, including Web-AI code-mode, code-extract, and adaptive-fetch routes.
  • Public SSE event types: 49 (src/core/bus.ts → src/core/event-bus.ts → GET /api/events).
  • Primary web event channel: GET /api/events SSE, with legacy WebSocket fallback only for pre-X-01 servers.
  • Slash command registry: 51 commands across CLI, Web, Telegram, and Discord, plus dynamic /skill:<id> for active skills (CLI/Web only).
  • Runtime registry: 13 top-level runtimes in src/cli/registry.ts.
  • Module scale: src/routes/ 36 TS files · src/agent/ 46 TS files · src/goal/ 5 TS files (includes pause-gate.ts) · src/browser/web-ai/ 96 TS files · src/browser/adaptive-fetch/ 34 files · src/telegram/ 5 files (includes hub-callback.ts) · src/manager/ 94 TS/TSX files (includes telegram-hub/).

Runtime Registry

src/cli/registry.ts is the single source for runtime keys, default models, and model choices. Current top-level runtimes are:

Runtime Role
pi Pi RPC runtime with isolated PI_CODING_AGENT_DIR profiles
agy Antigravity print-mode runtime (agy -p, optional --model when not default)
ai-e AI-E wrapper runtime
claude Anthropic Claude CLI
claude-e Claude E helper-backed runtime
codex OpenAI Codex CLI
codex-app Codex App stdio bridge
cursor Cursor Agent CLI runtime
gemini Gemini CLI runtime
grok Grok CLI runtime
kiro-code AWS Kiro CLI runtime
opencode OpenCode runtime
copilot GitHub Copilot ACP runtime

Wrapper runtimes such as ai-e, claude-e, and codex-app delegate to their underlying tools but remain first-class registry keys. agy, cursor, grok, and kiro-code are not AI-E providers; they are top-level runtime surfaces.

Request Flow

User prompt
  -> interface adapter (Web, CLI/TUI, Telegram, Discord)
  -> src/orchestrator/gateway.ts
  -> slash command parser or normal agent submission
  -> src/agent/spawn.ts
  -> provider CLI process
  -> src/agent/events/* adapter
  -> src/core/bus.ts broadcast
  -> SSE event bus + trace/tool-log snapshots + internal listeners

Important boundaries:

  • src/core/bus.ts publishes public events to src/core/event-bus.ts for SSE and still calls internal listeners for collectors/forwarders.
  • src/routes/events.ts exposes GET /api/events, a data-only SSE stream. Topic and event name are JSON fields in each data: payload.
  • public/js/event-channel.ts owns the browser EventSource singleton, exponential reconnect, Last-Event-ID replay, and fallback notification when a legacy server has no /api/events.
  • public/js/ws.ts still owns legacy WebSocket compatibility and event-specific UI dispatch.

Orchestration

CLI-JAW uses explicit orchestration for complex work:

I (Interview) -> P (Plan) -> A (Audit) -> B (Build) -> C (Check) -> D (Done)

Key points:

  • PABCD entry is explicit through jaw orchestrate, /orchestrate, or /pabcd.
  • Resume is explicit /continue; natural-language “continue/계속/이어서” is a normal prompt.
  • /plan is a compatibility guide for PABCD P, not a separate planning mode.
  • /review [focus] resolves a validated project directory from configured projectDirs or recent conversation/git evidence; it does not fall back to JAW_HOME or bare process.cwd().
  • /goal plan and /goalplan store a raw planHint and require /goal refine <specific objective> or cli-jaw goal refine ... before checkpoints are accepted.
  • /goal run ... is the bounded automation preview surface. Budget enforcement is tracking-oriented unless the corresponding gate has been implemented.
  • Worker progress is query-first through jaw worker status and watchable via jaw worker watch or jaw dispatch --watch.

Goal pause gate (P0):

  • src/goal/pause-gate.ts arms when an active goal has agentPauseCount ≥ 1 after the first audited agent pause.
  • While armed, automatic goal continuation is suppressed; the audit turn ends with goal_pause_gate_pending on the SSE channel.
  • A second audited agent pause or a productive checkpoint clears the gate. See structure/stream-events.md.

Pre-prompt context hooks:

  • src/prompt/context-hooks.ts injects bounded JSON snapshots from JAW_HOME/context-hooks.json before model reasoning (main / heartbeat scopes).
  • Inspected via jaw hooks inspect; kill switch CLI_JAW_PRE_PROMPT_HOOKS=0. Details: docs/dev/pre-prompt-context-hooks.md.

PABCD evidence gate:

  • Forward P→A/A→B/B→C/C→D requires jaw orchestrate <phase> --attest '{"from","to","did",...}' (C→D also checkOutput/exitCode). Goal mode self-advances but still requires attestation as proof-of-work.

API Surfaces

Use structure/server_api.md for the full table. Major route groups:

Group Examples
Core/system /api/health, /api/session, /api/runtime, /api/auth/token
Messages/sessions /api/messages, /api/messages/count, /api/chat-sessions
Events /api/events SSE
Commands /api/command, /api/commands, /api/message
Orchestration /api/orchestrate/* including dispatch, batch dispatch, state, worker progress
Goals/tasks /api/goal, /api/goal-run, /api/task
Runtime settings /api/settings, /api/cli-registry, /api/cli-status, /api/quota
Memory /api/jaw-memory/*, /api/memory/*, dashboard memory federation
Browser /api/browser/*, /api/browser/web-ai/*
Messaging /api/channel/send, /api/telegram/send, /api/discord/send
Manager /api/dashboard/board/*, /api/dashboard/schedule/*, manager-only routes
Jaw CEO /api/jaw-ceo/*
Traces/security /api/traces/*, /api/security-audit/*

/api/channel/send is the canonical outbound channel send endpoint. Telegram and Discord direct endpoints remain compatibility/direct paths.

Web UI Structured Renderers

The Web chat UI hydrates fenced structured payloads at final-render time (not during streaming) via public/js/render/* and src/shared/structured-fence.ts:

Fence / route Role
/api/link-preview Rich link unfurl cards (metadata + guarded image proxy)
search-results Native search-result cards from fenced JSON
compose-block Editable draft blocks (compose-block-v1 schema)
diff Unified diff viewer (explicit fence or auto-detect)
dataframe Sortable/filterable tables
chart-json Simple bar/line/pie charts
/media/:filename Inline media + lightbox

Telegram/Discord channels receive prompt guards so raw fences are not forwarded unchanged. See structure/INDEX.md rows 145–146 and structure/frontend.md.

Manager Dashboard

jaw dashboard serve runs a separate manager server on port 24576 by default. Electron implicit spawn uses a separate 24577-24590 lane.

The manager owns:

  • multi-instance discovery and cached InstanceRegistry scans,
  • live instance previews, worker SSE bridge (worker-sse-client.ts), and preview-origin proxying,
  • board, schedule, reminders, and connector surfaces,
  • Telegram Hub (P0–P4): forum-topic routing (chatId, threadId) → port, hub-member inbound proxy and outbound relay, per-topic model/systemPrompt overrides, Manager settings GUI (TelegramHub.tsx); hub commands /setthread /threads /hubhelp; one bot token → one long-poller invariant,
  • notes, WYSIWYG editing, graph/search, snippets, history, and assets,
  • git diff repo candidates, summary, and file diff APIs,
  • read-only dashboard memory federation (L1/L2) and optional embedding search,
  • Electron panel bridges for terminal, browser, diff, folder, docs, and Jaw CEO.

Manager routes are documented separately in structure/server_api.md because they are not all mounted on the core server.ts app.

Electron Desktop

The Electron app now ships as a self-contained desktop runtime instead of depending solely on a globally installed jaw binary. Packaged desktop builds include a Node.js sidecar:

  • scripts/bundle-sidecar.sh downloads Node.js 24.17 for the target platform.
  • The sidecar copies dist/, public/, package.json, and production dependencies into electron/sidecar/server.
  • Frontend-only dependencies are pruned before packaging.
  • better-sqlite3 is rebuilt against the bundled Node runtime.
  • A generated bin/jaw or bin/jaw.cmd shim launches dist/bin/cli-jaw.js.
  • electron/electron-builder.yml ships the sidecar as extraResources/server.
  • electron/src/main/lib/jaw-spawn.ts prefers the bundled sidecar jaw before falling back to JAW_BIN or a global jaw.

Current release targets:

  • macOS arm64: DMG + ZIP
  • Windows x64: NSIS installer + ZIP
  • Linux x64: AppImage

.github/workflows/desktop-release.yml builds these artifacts on GitHub Release publish and through manual workflow_dispatch.

Build And Verification

Common local commands:

npm run build
npm run build:frontend
npm test          # tsx --experimental-test-module-mocks tests/run.mts
npm run test:all  # same driver, all test files under tests/
npm run gate:all
bash structure/check-doc-drift.sh
bash structure/verify-counts.sh

npm test uses the programmatic tests/run.mts driver (isolation: 'process' per file for subprocess DB isolation) instead of a flat tsx --test glob.

Frontend TypeScript under public/js/**/*.ts requires npm run build:frontend. Backend TypeScript under src/**/*.ts requires npm run build or npm run typecheck.

Desktop packaging:

npm install
npm --prefix electron install
npm run electron:dev
npm run electron:dist:mac

npm run electron:dist:mac now runs frontend build, sidecar bundling, Electron build, and macOS packaging.

Documentation Sync

When command, API, orchestration, runtime, manager, or desktop behavior changes, update these together:

  • README.md
  • AGENTS.md
  • CLAUDE.md
  • structure/AGENTS.md
  • structure/INDEX.md
  • structure/server_api.md
  • structure/commands.md
  • structure/str_func.md
  • docs/ARCHITECTURE.md
  • docs/dev/pre-prompt-context-hooks.md when context-hook config or limits change
  • structure/telegram.md when Telegram Hub or topic routing changes
  • structure/stream-events.md when SSE event catalog or goal pause-gate events change
  • electron/README.md when desktop behavior changes

Run bash structure/check-doc-drift.sh before calling the docs current.