Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions openwiki/.last-update.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"updatedAt": "2026-08-06T08:57:36.644Z",
"updatedAt": "2026-08-07T08:31:06.133Z",
"command": "update",
"gitHead": "630eb9ec3fa22a4bed2d347fc3ea3a6a3bd22abc",
"gitHead": "a0e28a30fba1c80bc883711eab48292c5f8c398d",
"model": "z-ai/glm-5.2",
"status": "complete",
"language": "en"
Expand Down
6 changes: 4 additions & 2 deletions openwiki/agent/workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,10 +131,12 @@ Both subagents are read-only: they never create, edit, move, or delete files. Th

When a fatal signal fires with an active run, `handleFatal()` best-effort:

1. records the crash as a failure via `recordRunSafe()` so it appears in telemetry, classified and (when residual) fingerprinted by the same boundary as every other failure;
1. records the crash as a failure via `recordRunSafe()` so it appears in telemetry, classified by `describeErrorForTelemetry()` (a residual `agent_error` carries the innermost error's allowlisted name as its `error_detail`, the same fingerprint boundary as every other failure — see [Credentials and updates § Error classification and fingerprinting](../operations/credentials-and-updates.md#error-classification-and-fingerprinting));
2. stamps the run `interrupted` via `persistRunMetadataIfChanged()` so the next scheduled update retries instead of no-op'ing against a half-written wiki;
3. writes a local stderr line and, when `OPENWIKI_DEBUG` is set, the stack; then exits non-zero via `setImmediate`.

`handleFatal()` claims the active run synchronously, before step 1 and before any `await`: it calls `getActiveRun()` followed immediately by `clearActiveRun()` with no await between them. The installer fires one `void handleFatal(...)` per escaped rejection, and a burst of subagent rejections lands on the microtask queue together; reading and clearing with no await in between makes the claim atomic for the event loop, so the first handler owns the crash and every later handler sees `undefined` and only exits. Do not move any `await` above that pair — doing so reintroduces the race where every rejection records the same run and one crash produces hundreds of duplicate events (`test/crash-guard.test.ts`, "a burst of concurrent fatal signals records the crash exactly once").

Each side effect is wrapped and swallowed independently so a failure in one never blocks the other or the exit. The guard is the post-mortem counterpart to the catch block's interrupted-stamp path described under [Git evidence and update metadata](#git-evidence-and-update-metadata).

## Model errors
Expand Down Expand Up @@ -164,7 +166,7 @@ The same agent runtime is the wiki-generation backend invoked by the [DeepSWE ev
- `createAgentBackend()` wraps the wiki backend in an `OpenWikiCompositeBackend` (a `CompositeBackend` subclass) with `/skills/` and `/conversation_history/` mounts. The subclass overrides `glob` to convert a `RangeError` ("Maximum call stack size exceeded") from an over-broad pattern into a tool error so a runaway `**/*` glob no longer crashes the run (covered by `test/conversation-history-offload.test.ts`). `CONVERSATION_HISTORY_MOUNT` must stay in sync with deepagents' hard-coded `/conversation_history` default (there is no override); a dependency bump that moves that default silently reintroduces #496, so the offload test suite includes a drift probe that drives the installed `createSummarizationMiddleware` against a recording backend. Both mounts are denied to the model's filesystem tools via `AGENT_FILESYSTEM_PERMISSIONS`; do not loosen those deny rules without closing the prompt-injection path they guard.
- The live run streams via `agent.stream({ streamMode: ["messages", "tools"], subgraphs: true })` and is normalized by `parseAgentStreamChunk()`. `parseStreamEvent()` remains for the public agent factory's Agent Protocol v3 event shape only — if you change streaming, update `parseAgentStreamChunk()` and `test/stream-redaction.test.ts`, not the protocol parser.
- Init-only subagents are gated by `resolveSkeletonCriticSubagents()` and `resolveWikiQaSubagents()` (both `init` + `repository` only). Adding or changing the init verification loop means editing `src/agent/skeleton_critic.ts` / `src/agent/wiki_qa_subagents.ts` and the `CODE_SYSTEM_PROMPTS.init` template that drives them.
- The crash guard registers the active run only for the stream-consumption window; if you move streaming or add earlier fatal paths, ensure `registerActiveRun()`/`clearActiveRun()` still bracket the window a subagent rejection can escape through.
- The crash guard registers the active run only for the stream-consumption window; if you move streaming or add earlier fatal paths, ensure `registerActiveRun()`/`clearActiveRun()` still bracket the window a subagent rejection can escape through. Inside `handleFatal()`, the `getActiveRun()`/`clearActiveRun()` claim must stay synchronous (no `await` before or between them): it is the guard against a burst of escaped rejections producing one crash event rather than hundreds — asserted by `test/crash-guard.test.ts` ("a burst of concurrent fatal signals records the crash exactly once").

## Source map

Expand Down
8 changes: 5 additions & 3 deletions openwiki/architecture/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ After a non-chat run completes, `src/agent/utils.ts` computes a SHA-256 snapshot

The live run consumes the agent via `agent.stream({ streamMode: ["messages", "tools"], subgraphs: true })` rather than the Agent Protocol `streamEvents` API. `parseAgentStreamChunk()` in `src/agent/index.ts` normalizes each `[namespace, mode, payload]` chunk into an `OpenWikiRunEvent`: `tools` chunks become tool start/end events (tool-call strings pass through `sanitizeDiagnosticText()` so secrets are redacted), and `messages` chunks yield text after `extractMessageText()` filters non-text content blocks (`tool`, `reasoning`, `file`, `image`) so base64 payloads never reach the terminal. A namespace longer than one element marks the event `source: "subgraph"`, which is how init subagent output is attributed. `parseStreamEvent()` remains exported for the public agent factory's Agent Protocol v3 event shape, but the live run no longer uses it.

A process-wide crash guard (`src/agent/crash-guard.ts`) is installed once at CLI startup via `installCrashGuard()`. The run registers itself with `registerActiveRun()` for the stream-consumption window only and clears it in `finally`; if a rejection escapes every catch (e.g. a subagent error on the microtask queue), `handleFatal()` records the failure to telemetry, stamps the run `interrupted`, and exits non-zero. See [Agent workflow § Crash guard](../agent/workflow.md#crash-guard) for the post-mortem contract.
A process-wide crash guard (`src/agent/crash-guard.ts`) is installed once at CLI startup via `installCrashGuard()`. The run registers itself with `registerActiveRun()` for the stream-consumption window only and clears it in `finally`; if a rejection escapes every catch (e.g. a subagent error on the microtask queue), `handleFatal()` records the failure to telemetry, stamps the run `interrupted`, and exits non-zero. `handleFatal()` claims the active run synchronously — `getActiveRun()` followed immediately by `clearActiveRun()` with no `await` between them, before any async side effect — so a burst of escaped rejections landing together on the microtask queue produces one crash event: the first handler wins the run and every later handler sees `undefined` and only exits. Do not move an `await` above that pair; doing so reintroduces the race where one crash records hundreds of duplicate events. See [Agent workflow § Crash guard](../agent/workflow.md#crash-guard) for the post-mortem contract.

## Why the architecture is shaped this way

Expand All @@ -121,7 +121,7 @@ The current design reflects a documentation product rather than a general-purpos

- **OKF compliance** (`src/okf/`): `frontmatter.ts` validates and migrates YAML front matter, `index-labels.ts` localizes directory index headings by BCP-47 language, and `index-sync.ts` deterministically generates and synchronizes every `index.md` after a run. The OKF middleware (`src/agent/okf-middleware.ts`) ties these into the agent lifecycle.
- **Mermaid validation** (`src/mermaid/`): `fences.ts` extracts Mermaid code fences from wiki pages, `validate.ts` parses and validates them, and `wiki.ts` repairs broken fences by converting them to plain text fences with an HTML comment explaining the parse error. The OKF middleware calls `validateWikiMermaid()` after every run.
- **Telemetry** (`src/telemetry/`): emits a single `openwiki_run` PostHog event per run with mode, provider, outcome, latency, and configured connectors. `gates.ts` checks `OPENWIKI_TELEMETRY_DISABLED` / `DO_NOT_TRACK` for opt-out and uses `ci-info` to tag CI runs with a sentinel distinct ID so ephemeral runners never inflate install counts. `record-run-safe.ts` wraps the send with a 3-second flush timeout so telemetry can never stall the CLI. `errors.ts` classifies failures by walking an unwrap chain (`unwrapErrorChain()`, bounded at 5 links, cycle-safe) so a provider error hidden inside a tool-error wrapper or `AggregateError` is recovered instead of collapsing into the residual `agent_error` bucket. The origin-tag read is itself a chain walk: `readErrorOrigin()` mirrors `classifyError()` and returns the first link whose tag names an owned family (class + detail + throw-site stage), falling back to the nearest stage-only tag — so an owned error re-wrapped by a framework (LangChain's `MiddlewareError` copies the inner message but strands the tag on `.cause`) keeps its class instead of decaying to `agent_error`. The residual bucket also carries an `errorName` fingerprint via `innermostConstructorName()`, which walks to the deepest allowlisted constructor name (not the outer wrapper's) so a framework envelope does not collapse every distinct root cause to one name; the same `safeConstructorName()` allowlist keeps the anonymity envelope closed. `client.ts` `capture()` returns `true` only when the PostHog send fulfills before the flush timeout, so send failures and timeouts are reported as failures rather than silently swallowed.
- **Telemetry** (`src/telemetry/`): emits a single `openwiki_run` PostHog event per run with mode, provider, outcome, latency, configured connectors, and a build channel. `gates.ts` checks `OPENWIKI_TELEMETRY_DISABLED` / `DO_NOT_TRACK` for opt-out, uses `ci-info` to tag CI runs with a sentinel distinct ID so ephemeral runners never inflate install counts, and bakes a `BUILD_CHANNEL` (`"community"` in committed source, stamped to `"official"` only on the upstream release path — see [Credentials and updates § Build channel stamping](../operations/credentials-and-updates.md#build-channel-stamping)) so every event carries it and the dashboard can filter fork-originated telemetry. `record-run-safe.ts` wraps the send with a 3-second flush timeout so telemetry can never stall the CLI. `errors.ts` classifies failures by walking an unwrap chain (`unwrapErrorChain()`, bounded at 32 links, cycle-safe) so a provider error buried under several framework envelopes is recovered instead of collapsing into the residual `agent_error` bucket, with one origin-tag override (`streamOpenDisguisesProvider()`) that reclassifies a `build_error/stream_open` tag masking a provider failure. The residual `agent_error` bucket's one signal is the innermost error's own allowlisted name, folded into `error_detail` via `innermostErrorName()`; see [Credentials and updates § Error classification and fingerprinting](../operations/credentials-and-updates.md#error-classification-and-fingerprinting) for the full taxonomy, identifier allowlist, and override rules. `client.ts` `capture()` returns `true` only when the PostHog send fulfills before the flush timeout, so send failures and timeouts are reported as failures rather than silently swallowed.
- **Skills** (`src/agent/skills.ts`): bundles the `skills/` directory into the OpenWiki home and exposes it to the agent as the `/skills/` virtual mount on the `CompositeBackend` built by `createAgentBackend()` (see [DeepAgents backend and middleware](#deepagents-backend-and-middleware)). Write access to `/skills/**` is denied to the model via `AGENT_FILESYSTEM_PERMISSIONS`. Each bundled skill is staged in a unique scratch directory and swapped into place with an atomic `rename`, so repeated or overlapping `--init` syncs are idempotent — a concurrent install that lands first is accepted as success rather than racing with `EEXIST` or `ENOTEMPTY` errors.
- **Diagnostics and redaction** (`src/diagnostics.ts`): redacts secrets from error messages, headers, and provider responses before they are shown to the user or written to logs. It matches exact secret values from the environment and known token shapes (`sk-…`, `Bearer …`, `ls…`).

Expand Down Expand Up @@ -159,7 +159,7 @@ The current design reflects a documentation product rather than a general-purpos
- `src/diagnostics.ts`
- `src/okf/frontmatter.ts`, `src/okf/index-labels.ts`, `src/okf/index-sync.ts`
- `src/mermaid/fences.ts`, `src/mermaid/validate.ts`, `src/mermaid/wiki.ts`, `src/mermaid/dom-shim.ts`
- `src/telemetry/` (including `errors.ts`, `record-run-safe.ts`, `client.ts`)
- `src/telemetry/` (including `errors.ts`, `gates.ts`, `record-run-safe.ts`, `senders.ts`, `client.ts`)
- `src/auth/oauth.ts`
- `src/auth/providers.ts`
- `src/auth/configure.ts`
Expand All @@ -174,3 +174,5 @@ The current design reflects a documentation product rather than a general-purpos
- `src/code-mode.ts`
- `src/constants.ts`
- `package.json`
- `scripts/stamp-build-channel.cjs`
- `.github/workflows/release.yml`
13 changes: 12 additions & 1 deletion openwiki/operations/credentials-and-updates.md
Original file line number Diff line number Diff line change
Expand Up @@ -229,10 +229,18 @@ Enforcement and bounds:

## Anonymous usage telemetry

OpenWiki collects anonymous, per-machine usage telemetry via PostHog (`src/telemetry/`). The system emits a single `openwiki_run` event per run with mode (code/personal), provider, outcome (success/failure), latency, environment, and configured connectors. Telemetry can be disabled by setting `OPENWIKI_TELEMETRY_DISABLED=1` or `DO_NOT_TRACK=1`.
OpenWiki collects anonymous, per-machine usage telemetry via PostHog (`src/telemetry/`). The system emits a single `openwiki_run` event per run with mode (code/personal), provider, outcome (success/failure), latency, environment, configured connectors, and a `build_channel` stamp. Telemetry can be disabled by setting `OPENWIKI_TELEMETRY_DISABLED=1` or `DO_NOT_TRACK=1`.

CI and scheduled runs are detected via `ci-info` (or `OPENWIKI_SCHEDULED=1`) and sent under a sentinel distinct id per provider rather than the machine's install id, so ephemeral CI runners do not inflate human install counts. The install id is stored at `~/.openwiki/install-id` and a one-time disclosure notice is shown on first run (`src/telemetry/config.ts`). Telemetry never stalls a run — the send and client shutdown are bounded by a 3-second flush timeout (`src/telemetry/config.ts`).

### Build channel stamping

Every event carries a `build_channel` property (`"official"` or `"community"`) baked into the build so fork-originated telemetry can be filtered from the official-release signal. The committed default in `src/telemetry/gates.ts` is `"community"`; the upstream release pipeline rewrites that one `BUILD_CHANNEL` assignment to `"official"` via `scripts/stamp-build-channel.cjs` (driven by the `OPENWIKI_BUILD_CHANNEL` env var set in `.github/workflows/release.yml`), so only npm-published upstream builds report `"official"` and every fork, local build, and source/dev run reports `"community"`. The stamp is fail-safe: an unset or unrecognized value always resolves to `"community"`, so an unexpected env value can never mint an `"official"` build, and the stamp throws if the expected single `BUILD_CHANNEL` assignment is not present exactly once (so a drifted file fails the release loudly instead of silently publishing an unstamped build). The rewrite is ephemeral in CI (a throwaway checkout that is never committed back), so the committed source stays `"community"`. The stamp runs inside the `pnpm release` script (publish path only, before `tsc`), never on the version-PR path.

### Error classification and fingerprinting

Failure events are classified by walking an unwrap chain (`unwrapErrorChain()`, bounded at 32 links, cycle-safe) so a provider error hidden inside a tool-error wrapper or `AggregateError` is recovered instead of collapsing into the residual `agent_error` bucket. The origin-tag read is itself a chain walk: `readErrorOrigin()` mirrors `classifyError()` and returns the first link whose tag names an owned family (class + detail + throw-site stage), falling back to the nearest stage-only tag — so an owned error re-wrapped by a framework keeps its class instead of decaying to `agent_error`. The one override is `build_error/stream_open`: that stage is the first provider round trip, so a failure there carrying a provider signal (the raw classifier already naming it `provider_error`, or an HTTP status on the chain paired with any non-residual class) is a disguised provider error and the raw classification wins over the tag, landing the failure on the provider instead of being counted as our build bug. The residual `agent_error` bucket carries no fixed detail; its `error_detail` is the innermost error's own allowlisted name (`innermostErrorName()`), read from both `.name` (which a framework like LangChain's `MiddlewareError` copies up from the inner error) and `constructor.name`, walking to the deepest link so a framework envelope does not collapse every distinct root cause to one name. The identifier gate (`isSafeErrorIdentifier()` in `src/telemetry/taxonomy.ts`) allows only a bare ASCII identifier (letters/digits with single interior underscores, ≤64 chars); anything else is dropped so the anonymity envelope stays closed. The `errorName` field was removed — the residual bucket's signal now travels in `error_detail` — so every failure class reports a single shared detail property.

## Scheduled CI workflows

During `openwiki code --init`, `src/code-mode.ts` also creates `.github/workflows/openwiki-update.yml` in the target repository if it does not already exist. On `--update` and chat runs, an existing workflow file is preserved verbatim so repo-specific customizations (fork guards, pinned actions, custom steps) are never silently overwritten. AGENTS.md and CLAUDE.md snippets are refreshed in place on every code-mode run using `<!-- OPENWIKI:START -->` / `<!-- OPENWIKI:END -->` markers.
Expand Down Expand Up @@ -284,6 +292,7 @@ Bitbucket users should configure repository variables for the model provider key
- The content-snapshot check means CI runs that produce no changes will not update `.last-update.json` or open a PR with metadata-only changes.
- Scheduled update workflows must fetch full history (`fetch-depth: 0` for GitHub Actions, `GIT_DEPTH: "0"` for GitLab CI, `clone: depth: full` for Bitbucket). A shallow clone hides the commit recorded in `.last-update.json`, so `openwiki code --update` cannot build a change window and runs against an empty summary.
- Interrupted runs write `status: "interrupted"` so the next update retries. If metadata semantics change, keep `getUpdateNoopStatus()` and `persistRunMetadataIfChanged()` in sync so the interrupted/complete lifecycle is preserved.
- The `build_channel` stamp (`scripts/stamp-build-channel.cjs`) targets exactly one `const BUILD_CHANNEL: BuildChannel = "…"` assignment in `src/telemetry/gates.ts`. Renaming that line, splitting it, or changing its formatting breaks the regex and fails the release loudly (`test/stamp-build-channel.test.ts`). Keep the committed value `"community"`; only the upstream release pipeline (`.github/workflows/release.yml`) sets `OPENWIKI_BUILD_CHANNEL=official`. A drifted `gates.ts` that no longer matches the assignment pattern will throw instead of silently publishing an unstamped build.

## Source map

Expand All @@ -296,6 +305,8 @@ Bitbucket users should configure repository variables for the model provider key
- `src/external-cli-auth.ts`
- `src/diagnostics.ts`
- `src/telemetry/`
- `scripts/stamp-build-channel.cjs`
- `.github/workflows/release.yml`
- `src/auth/oauth.ts`
- `src/auth/providers.ts`
- `src/auth/configure.ts`
Expand Down
Loading