Skip to content

Latest commit

 

History

History
179 lines (157 loc) · 11 KB

File metadata and controls

179 lines (157 loc) · 11 KB

TODO — what is knowingly not done

Gaps, accepted trade-offs and deferred work. Nothing in README.md or DESIGN.md may imply a control or a feature that only this file knows is missing.

Unverified seams (Phase 0 — do this before trusting anything below it)

The monitor is load-bearing twice over: it hosts the worker and it delivers. All of its seams — manifest schema, environment, lifetime, and the wake — have now been verified against the installed CLI, live.

  • The monitor manifest schema. Read out of the installed CLI (v2.1.220): monitors/monitors.json is parsed as a bare array of strictObject entries — name, command, description required, optional when ("always" | "on-skill-invoke:<skill>"), names unique within the plugin. This caught a real bug: the file was written as {"monitors": [...]}, which fails the whole plugin's monitor load — and since the monitor hosts the worker, it would have failed silently and completely. Locked in by a test.
  • A plugin-declared monitor receives CLAUDE_CODE_SESSION_ID in its environment — verified live by reading /proc/<pid>/environ of running plugin-monitor workers (it also receives no CLAUDE_PLUGIN_DATA, which is why paths.data_dir() derives it).
  • A plugin monitor survives for the whole session and can host a long-running asyncio process — verified across multi-day live sessions. The once-planned fallback (a hook-spawned detached worker, spawn.py) was implemented, proved strictly worse, and removed: the monitor is the only host, headless surfaces are out of scope, and a monitor dying mid-session means no observer until the next session (accepted).
  • A monitor stdout line actually wakes a stopped session — verified live (2026-07-30): an advisory pushed two seconds after a turn ended arrived as a task-notification that re-opened the stopped loop with no user input; the wake turn's own turn-end pass then produced an organic advisory that woke it a second time, and the chain self-terminated via dedup. The finish gate's deferred half rests on this seam, now proven.
  • SubagentStop carries last_assistant_message on the installed CLI — verified live, including the trap it carries: the harness also fires it for anonymous UI helper agents (prompt suggesters) whose "final message" is a proposed user prompt; those are dropped by the missing agent_type.
  • additionalContext on PostToolUse reaches the model without blocking — verified live from the first delivered advisory onward.

Every Phase-0 seam is now verified against the installed CLI, live. Both delivery channels are proven, including the monitor wake of a stopped session.

Not implemented

  • Resumed-task digest on SessionStart. A resumed task's ledger is loaded by the worker but never surfaced to the primary (Phase 5 in the design).
  • Staleness check on resuming a long-dormant task. Open question #9: a task resumed after days resumes against a repo that moved underneath it, and the ledger's cited locators are not re-validated.

Accepted trade-offs

  • A slash command always costs a model turn. Claude Code commands are prompts: the ! block runs, its output is injected, and the model is invoked to relay it. The frontmatter schema has no opt-out (verified against the installed CLI — disable-model-invocation is the inverse, stopping the model from invoking the command). So /second-brain-stats spins the primary loop, which sits awkwardly beside the design's claim that the human surfaces cost nothing. The statusline and ! bash mode are the model-free paths, and both are documented in the README.

  • The worker holds its code until the session restarts. It is a long-lived process started at session start, so editing anything under worker/ has no effect on the running one — /reload-plugins re-arms commands and hooks but does not restart the monitor's process. Symptom seen live: /second-brain-run appeared to work while the request file was never read, because feeding the spool tripped the ordinary volume trigger instead. The command now says so when it detects an older worker, and a request expires after five minutes so a stale one cannot fire a pass later. Restart the session after changing worker code.

  • The mcp SDK is an optional dependency. mcpclient.py is a thin adapter over the official SDK — one session per server, shared across forks — but the SDK is not vendored, so on a machine without it every MCP-granted tool is absent. That is reported (a warning, a status field, /second-brain-stats) rather than silently reducing a detector's reach, which is the failure mode that matters: a detector answering confidently without the tool it asked for is worse than one that does not answer. Untested against a live server.

  • Transport is ours, not the official SDKs. DESIGN.md §What runs the loop argues for the provider SDKs as typed HTTP clients. A Claude Code plugin cannot assume it may install packages and the worker has to run on a bare python3, so provider.py builds requests against the published wire formats and posts them with httpx when present and urllib on a worker thread otherwise (http.py). What that costs, concretely: no streaming, and retries/backoff are ours (two retries on 429/5xx). Revisit if the plugin ever ships a vendored dependency set.

  • The pilot's cache-warm signal is completion, not first token. Without streaming there is no earlier signal, so the fan-out waits for the pilot fork to finish rather than for its first streamed token. Costs latency on every pass, never correctness.

  • The window is not reconstructed from the transcript after a crash. The design notes the projection is deterministic and therefore replayable; the worker actually restarts from the ledger plus new observations. A restart costs the warm prefix and the uncompacted tail, not just the cache.

  • Task splitting is a keyword heuristic, not a model judgment. task.py uses prefix/marker matching rather than asking the model whether a prompt is a new goal. Deliberately wrong in the cheap direction (an extra split costs a cold start); open question #3 asks whether it misfires often enough to matter.

  • static-analysis ships with an empty command allowlist. It cannot infer a project's build command, and guessing one would be the one detector able to execute guessing wrong. The workspace supplies it.

  • Uptake is self-reported. Constrained by evidence-or-nothing and a default of no_evidence, but a model grading its own advice is structurally flattering. Open question #2: is it honest enough to tune the gate on?

  • Per-detector budgets are enforced per fork, not globally. Two concurrent sessions each enforce their own token ceiling; there is no cross-session budget. The fix, if it is ever needed, is a lockfile-guarded counter — not a service.

Known gaps

  • claude --plugin-dir development can still split state. The data-dir derivation (paths._derived_data_dir) heals the missing CLAUDE_PLUGIN_DATA for monitors and ! command blocks only when the plugin runs from the install cache; a source checkout falls back to data/second-brain, so if the harness hands hooks a differently-named dev directory, the two-world split returns in dev mode. Fix if it ever bites: a pointer file written by the component that has the env.

Known drifts from a full design-vs-implementation review (2026-07-30)

Confirmed against the code, accepted for now, in rough severity order:

  • Cache economics on the default model warm up slowly. Haiku 4.5's minimum cacheable prefix is 4096 tokens, so a short task's passes can all report zero cache activity while behaving correctly (DESIGN.md §What the cache actually does). The OpenAI wire additionally never reports cache writes — implicit caching has nothing to declare — so cache_write is 0 there by construction.
  • Budget accounting has three leaks: the per-task budget is never reset at a task rebind (it actually bounds the worker's lifetime — conservative direction); compaction's model call discards its usage; a hard-deadline-cancelled fork returns zero usage, so the most expensive forks are the uncounted ones.
  • The Edit line anchor is largely inert: it resolves old_string against the post-edit file, so a normal replacement anchors nothing and a re-occurring string can anchor the wrong line.
  • Two path-jail gaps: Grep opens walked files without re-resolving symlinks (an in-tree symlink file pointing outside the workspace is readable); Glob's escape protection works by a relative_to exception rather than by design.
  • Staleness and mute are pass-granular: the drain hook checks expiry clocks only, so an action after the last pass cannot kill a queued advisory, and a mute set while advice is queued does not stop its delivery. A finish-gate advisory minted before /clear can still block a stop afterwards.
  • repeat-failure is a model judgment, not the structural worker-side trigger DESIGN.md §What a detector sees claims; only cross-task-collision is structural.
  • Rate-limit slots are charged at gate time, in memory — expired advice still burns a slot, and a worker restart resets the hour window.
  • The envelope's adjudication fields are dead (delivered_at/channel/ verdict on the dataclass); delivery and verdicts live in spool META records and the ledger instead.
  • The gate flowchart in DESIGN.md orders the stages differently from the code (code runs evidence → mute-free → suppression → dedup → floors, which is the saner order — dedup before the confidence split stops a repeated sub-floor hunch from spamming the user). The diagram, not the code, is what should change.

Catalogue status

Shipped enabled: default, repeat-failure, standard-questions — the design's shipping order, stopping where tools begin. Defined and disabled: git-log, prior-art, constraint-drift, goal-drift, cross-task-collision, static-analysis. Each is one /second-brain-config away, and none has been calibrated against real sessions yet, which is the point of shipping them off.

Open questions

Carried from DESIGN.md §Open questions, unchanged by the implementation — the first is still the only one that decides whether this is worth having:

  1. Does it actually produce better outcomes? The honest test is a month of long tasks and the question "would you turn it off?"
  2. How honest is self-adjudication?
  3. How often does task-boundary detection misfire?
  4. Would a detector ever need enough agentic depth to justify losing the shared prefix?
  5. Should subagents be watched, or only their conclusions?
  6. How loud should collision warnings be?
  7. How wide can the fan-out go before provider rate limits bind?
  8. Does a cheap model have the judgment for this at all?
  9. What happens to a task that spans days?