Skip to content

PROGRESS.md

Latest commit

 

History

History
908 lines (775 loc) · 74.2 KB

PROGRESS.md

File metadata and controls

908 lines (775 loc) · 74.2 KB
Raw

RuvNet Brain — Build Progress (living tracker)

Updated: 2026-07-27 late-morning EDT · honest status, no overclaiming. "DONE" means proven with pasted evidence.

2026-07-27 — the honesty gate was reading a rotting artifact — v3.9.90-dev (branch fix/claims-artifact-freshness)

Two public surfaces were lying, and the same defect produced both. The README coverage badge said 26% while the ledger re-derives 28%, and 149,930 source chunks outlived the real 150,161 on four surfaces (README, explainer/index.html, llms.txt, llms-full.txt) — plus "54 public stores" (really 62) and README's "57 built stores" (really 69).

Root cause, measured, not guessed. claims-verify.mjs graded the coverage claim from coverage/coverage-summary.json without ever asking whether that file was still true. It was nine days stale (mtime Jul 18). Worse, vitest's default is reportOnFailure: false and its provider does if (!this.options.reportOnFailure) await this.cleanAfterRun() — so any run with a failing test DELETES the coverage directory and writes nothing. Reproduced live here: a complete run, 5 failures, coverage/ gone. One unvalidated rottable input, three possible lies: a false FAIL, a false PASS, a crash. Same class as #46 — ceremony wearing the costume of substance.

Fixed at cause, both halves. (1) reportOnFailure: true — a red run still produces a complete measurement, because whether a number may be QUOTED is a decision for the gate, not something to settle by deleting the evidence. (2) The coverage claim is now UNVERIFIABLE unless the artifact is present, COMPLETE (an entry for every file coverage.include covers — a fragment's total is a denominator nobody measured) and NEWER than every source in that set and the config that defines it. Unverifiable ⇒ a loud SKIPPED (not a pass) naming npm run test:cov, never a derived number. The source set is imported from vitest.config.mjs rather than hand-copied, so it follows the globs. mtime over a hash sidecar, deliberately: a hash this script wrote would be self-certifying.

One writer, one pass. npm run claims:fix re-derives the census from the idmaps and the % from the summary and stamps README + all four surfaces together; the gate stays read-only by default, and the writer obeys the same freshness precondition (it will not propagate a rotten number). count-chunks.mjs now delegates to it — a second writer with its own regexes is how four surfaces drift four ways.

Proof. Red-first, verbatim in the test headers: the decisive red was expected 'PASS' to be 'SKIP' — the stale artifact GRADED, and graded as a pass. Mutation test re-breaks the precondition and proves the same fixture goes back to PASSing. Also found: tests/mutation/*.test.mjs was in vitest's include list but no npm script or CI step ever ran it — both mutation suites had never once executed in CI. Now npm run test:mutation, wired into ci.yml.

Honest coverage: 28% = floor(min(statements 29.1, branches 28.62, functions 34.87, lines 30.67)) over 117 first-party source files, from a complete 2,009-test run. Thresholds unchanged — every measured value already clears them; raising them to a laptop-measured floor would risk a red CI for no honesty gain.

2026-07-26 (afternoon+evening) — CI green after 5 red days; remote CI becomes a ship gate; the ADR-053 duel — v3.9.78→82-dev

The streak: ci had been red since 07-21 (~70 runs, six releases shipped past it — nothing on the ship path ever asked the remote verdict). Fixed in three unmasking rounds, each layer hiding the next: (1) 9 win32 root causes across 11 files + 2 machine-bound tests + 5 REAL product portability bugs (memory-doctor's WAL-sidecar leak on ubuntu's sqlite 3.45.1 — mode chosen up front from sidecar presence now; wired-check OS-dependent path keys; capability-registry backslash-doubled Windows commands; distill-project .cmd spawn; MSYS pwd in the ledger); (2) HOME-only fixtures (win32 reads USERPROFILE) + coverage honestly re-derived 14→26%; (3) the installer smoke that had not RUN in days under the red streak (canonical-ledger shadowing, argv-array plist assertion, fake-ruflo shim). Both CI jobs green on a49e2e2 and every commit since.

Gate C++ (v3.9.81): release.mjs now refuses to publish on a red or UNKNOWN remote CI — ci-verdict.mjs pure decision + 9-case contract test where the refusals are the load-bearing rows.

The ADR-053 duel (F5 fresh-context × GPT-5.6-Sol via codex, identical adversarial briefs): converged independently on six verdicts — full record in the ADR, now Accepted. LIVE FIND by both: three unprompted-speech hooks shipped with NO timeout (60s default on every prompt/write, 12× the /rvbc hang) — fixed same-commit (v3.9.82) + hook-contract now REFUSES missing timeouts and caps prompt-path at 5s. Rollout list 1–10 in the ADR; item 1 shipped, hook battery v2 is next.

Also: #43 closed with published-bytes proof (morning); local main reconciled, 4.0 dial WIP restored conflict-free; gh token gained workflow scope; 5 machine-wide lessons promoted to global memory; feat/issue-attack preserved on origin awaiting owner review. OPEN: John Losko's newbug.md ("one line missing somewhere in kb") — file not yet received; kb scripts parse + E2E query green locally, so not a universal breakage.

2026-07-26 — Issue #43: Codex wiring was dead on every npm install (Henrik Pettersen) — v3.9.77-dev

The bug: the npm tarball's files whitelist excluded plugin/mcp/server.mjs — the exact package-relative path wireCodexHost() resolves — so every npx ruvnet-brain from the registry hit no-source and Codex never got wired. Worked from a repo/marketplace checkout only; every existing codex test ran against the source checkout, the one place the file always exists.

The fix (all per #43's acceptance, verbatim): files ships exactly plugin/mcp/server.mjs (+ !plugin/README.md for npm's always-include rule — caught live by the new gate's first run); both Codex writes are atomic via atomicReplace (write-beside + rename, symlink-identity and file-mode preserving, recursive temp cleanup); new tests/unit/npm-tarball-codex.test.mjs runs npm pack, unpacks the real tarball, wires from the ARTIFACT, speaks real MCP initialize/tools/list to the installed server, and injects write failures cross-platform (directory-squat on the deterministic .tmp-<pid> path — no chmod, which is a win32 no-op).

QA: mutation-proven both ways (files entry removed → 8/8 red reproducing no-source verbatim; chmod-preserve removed → mode guard red). Independent opus review found the symlink/mode clobber + unguarded cleanup + RUVNET_BRAIN_KB test flake — all fixed same session. Full unit suite 1702 green; plugin battery 60/60 (warmed cache). Also unblocked the ship path itself: ADR-0042's supersede line was unparseable by the ADR gate on origin/main (would have blocked ANY release).

Channels: npm 3.9.77-dev published + latest; origin/main pushed. GitHub Release moves with tonight's nightly by design (2026-07-24 decision: self-update.mjs is the only release cutter; its issue-#29 script-drift rule fires on bin/install.mjs). ADR-051 addendum records all of it.

2026-07-24/25 — The night the pipeline's silence broke (issues #38/#39/#41/#42 + automation redesign) — v3.9.72-dev

Trigger: Stuart learned of four open issues from a GitHub email — 28h after the first was filed — and asked: "Are you set up to automatically fix issues and be aware of them?"

The answer was the incident: detection worked (issue-fix every 30m, issue-watch hourly), but the fixer's comments post as stuinfla and the watcher counted any stuinfla comment as "owner responded" → SLA pages structurally impossible. Its failure-comment dedup was also dead (attempt- start state write erased failureCommentAt) → 22 public bot comments on #38 while tkmeownow's tested patch sat unread in comment 1. The session banner only surfaced breaches → same poisoned predicate → silent. Score of the auto-fixer on the four issues: 0/4.

Shipped (all same night, v3.9.72-dev):

  • #38 Windows: hook-shim-bash.mjs resolver (env override → Git-for-Windows → where.exe minus WSL System32), skip-with-once-only-notice, fileURLToPath PLUGIN_ROOT. Built on @tkmeownow's patch, credited. Residual: lesson-hooks.sh/anticipate.sh bypass the shim.
  • #39 ruvector.db: proven chain — external ruvector mcp-server boots IntelligenceEngine at module load → initVectorDb() passes NO storagePath → native default ./ruvector.db at cwd = user's open project. Fixed with node -e launcher relocating cwd to ~/.cache/ruvnet-brain/ ruvector-mcp (no ruvector flag exists — verified). Signature-guarded cleanup for victims. Upstream storagePath plumb-through = OPEN offer to file against ruvector.
  • #41 gate false positives: @sparkling's shellSkeleton in hook-input.mjs (ADR-0021), skeleton verb, both MATCH_RE branches; probe table = permanent regression suite; true positives still block. The live buggy gate blocked our own fix agent's quoted command mid-work.
  • #42 Codex wiring: install-time [mcp_servers.ruvnet-brain] managed block in ~/.codex/ config.toml (merge-never-clobber, idempotent), native plugin skills discovered through the real Codex loader, leaked /Users/ path removed + repo-wide guard test, doctor probes Codex wired/not-wired. The obsolete repo-local skill.toml surface was removed on 2026-07-28.
  • ADR-050 issue-pipeline invariants: bot comments never satisfy SLA (BOT_MARKER exclusion), first-sighting ntfy page (awareness ≤1h), one failure comment per issue EVER, circuit breaker (2 fails → stop until issue changes + urgent needs-human page), banner always shows open count. 12 regression tests, 8 provably fail on old code (stash-mutation proof).
  • Security sweep mandate (new standing order): aidefence scan on issue bodies (4/4 benign; 3 lexical false positives adjudicated), fixer prompt hardened (title as JSON-escaped data, data-not-instructions, refuse gate-weakening patches), security review pre-ship.

Cross-model eval (Fable 5 × GPT-5.6 duel): converged on 3 root causes — single-environment development shipped as universal; health surfaces asserting unprobed paths (doctor "works in EVERY project"/"drops zero files" while Codex unwired + cwd pollution); gates without adversarial corpora. GPT-5.6 uniquely caught: windows CI job claims "zero exclusions" while 4 suites skipIf(win32) — a false coverage claim. Scores: F5 58 vs GPT 32 → converged verdict in ADR-050. Structural prevention set: consumer-contract OS×host matrix, quote-aware gate parser corpus, identity-separated triage-first automation.

Ops notes: issue-fix launchd job unloaded during redesign, reloaded post-ship with fixed code. Other session shipped 3.9.71 (console rebuild) mid-evening; its harness reaped this session's first 4 agent worktrees (~10 min lost) — manual scratchpad worktrees were the fix. ntfy channel HTTP-200-verified with a labeled test push; phone-side receipt = Stuart to confirm.

2026-07-13 — Every scheduled job must PROVE it ran. And MetaHarness stops being decorative.

Two of Stuart's hardest calls of the project, both fair, both acted on the same night.

1. "The meta harness isn't doing anything for me."

He was right, and the receipts proved it: 3 dispatches in the router's entire life, all of them my own test pings, $0.018 saved — while real work ran inline on the most expensive model available. Two faults:

  • Behavior. SKILL.md said "consult the engine" — advisory, and advisory rules get ignored. Now a HARD RULE with a taxonomy (mechanical→haiku · analytical→sonnet · judgment→main loop · pure-text→ OpenRouter) and a mandatory self-audit: doing mechanical work inline is a DEFECT you name out loud.
  • Measurement. A Claude Code subagent INHERITS the main-loop model unless overridden — a 5-agent fan-out on a Fable session is 5 Fable agents. That is the biggest cost lever there is, and NOTHING LOGGED IT (route-cheap only logged OpenRouter calls), so right-sizing was invisible even when it happened. scripts/dispatch-receipt.mjs now receipts subagent dispatches against the model they WOULD have inherited (the honest counterfactual). Live pricing, verified from the OpenRouter API: fable-5 $10/$50 · opus-4.8 $5/$25 · sonnet-5 $2/$10 · haiku-4.5 $1/$5 — mechanical work on Fable costs 10x the same work on Haiku. Sizing prefers MEASURED tokens over chars/4 (a file-reading agent burns ~20x what its prompt implies; estimating from the prompt understates savings so badly routing looks pointless). Logger AND viewer now ship (npm + installer) — a router with no scoreboard stays decorative.

Result, same night: 5 real dispatches, ~$2 saved (vs $0.018 in the router's whole prior life) — and they found real bugs, which is the point. Model guidance for Stuart: Opus 4.8 as the main loop; Fable is 2x the price for work Opus does as well. The leverage is in what gets SPAWNED.

2. "You told me the gong strategy fixed this. Were you lying?"

Fair, and the honest answer is that the effect was the same as a lie: the gong system covered the BRAIN going dark and DEAD KEYS — never job liveness — and I let him believe it was total.

THE TRAP, stated plainly because it defeats the obvious design: launchctl reports last exit status 0 for a job that has NEVER RUN — byte-identical to a job that ran and succeeded. "Check the exit code" cannot distinguish triumph from total absence. Silence was being read as health.

The fix is mechanism, not per-job discipline (discipline rots):

  • config/scheduled-jobs.json — the registry of what MUST run. Reality is compared against it, so unloaded / deleted / never-fired is a violation, not a silence.
  • scripts/job-heartbeat.sh — wraps every job: start receipt, end receipt, real exit code (still exits with it, so launchd sees truth), urgent ntfy on failure. Break-testing caught a real defect in my own wrapper: a POSIX shell blocked on a FOREGROUND child does not run its trap when signalled, so a killed job left a stale "running" receipt — the exact silent death it exists to prevent. Fixed by backgrounding the child and waiting. Verified across all four death modes: ok→0 · fail→7 · SIGTERM→143 · SIGKILL→uncatchable by definition (caught one level up).
  • scripts/nightly-watchdog.mjs — MISSING · NEVER-RAN · STALE · FAILING · OK. Transition-only gongs. It runs through the wrapper and sits in its own registry: a supervisor nobody supervises just moves the blind spot up one level.

All 7 jobs now produce fresh, successful receipts. com.ruvnet.npm-token-renew was 100% BLIND — on a healthy day it wrote ZERO BYTES anywhere, so "ran fine" and "never ran" were indistinguishable ~76 days out of 90. The wrapper cured it without touching a line of its logic. Retired 5 dead Ask-Ruvnet LaunchAgents (a Sonnet subagent traced all three failures to ONE defect: kb-evergreen.mjs hardcodes a @claude-flow/embeddings path the installed CLI no longer ships; pipeline-health had been exiting 1 correctly every morning since 07-09, screaming into a void where nothing listened).

MY ERROR, ON THE RECORD. I told Stuart brain-nightly had "never fired." Wrong. It fired at 03:15 on 07-12 and crashed three times — logs/nightly.err.log, mtime 03:15: /bin/bash: logs/nightly.log: No such file or directory. It died before it could write its own log, and I read that empty log as "never ran." That inference is precisely the trap the watchdog exists to catch, and I walked into it while building the watchdog. Absence of evidence is not absence of an event — it is frequently the evidence of a crash. Root cause: no WorkingDirectory in the plist until 13:39 that day, so it ran from /. Fixed. Tonight's 03:15 run PROVES it with a receipt instead of me inferring anything.

2026-07-12 (late night) — EVERY workflow green: Windows becomes a real gate; gists-nightly resurrected; injection hole closed

Stuart's QA mandate ("figure out why everything you call perfect keeps crashing at GitHub") answered with a full audit of ALL workflows to per-job conclusion — which immediately found reds the ci-only view had missed. Commits 4a8e0db → 6a2db75 → 51eb434 → f1b1339, each verified on the remote runner before the next claim.

windows-unit: 6 files / 17 tests failing → GREEN → BLOCKING. The dated note said "two bugs"; the runner's log said six clusters. All fixed at root cause, none papered over: (1) new URL(...).pathname → fileURLToPath() (the file:///D:/D:/ doubling), (2) /bin/bash → PATH bash (= Git Bash on runners, the same shell Claude Code hooks use on real Windows), (3) SHIPPED-code bug in sync-version.mjs — path.relative() backslashes vs a forward-slash EXEMPT set made its stray-literal gate unpassable on every Windows machine, (4) tests overrode HOME but os.homedir() reads USERPROFILE on Windows → consent state bled between tests through the runner's real profile, (5) no sqlite3 CLI on Windows runners → describe.skipIf(!hasSqlite3) gated on the CLI not the OS, (6) chmod-read-only is a no-op on win32 → that one scenario skipIf'd with the reason in code. First green ever: run 29219328821; continue-on-error removed the same night and the gate-flip run (29219425277) passed with windows-unit REQUIRED.

gists-nightly: born broken (3/3 runs failed since 07-10), now green. GITHUB_TOKEN is a GitHub App token and the gists API is closed to App tokens entirely — no permissions key can fix it. listGists now falls back to the unauthenticated public API (proven live: 438 gists via fallback with gh stripped from PATH); a shared-IP rate limit in --index-only exits 0 with a loud SKIP (self-heals next night). Dispatched run 29219196767 = first success in the workflow's life. The integration test guarding the old "gh failure = fatal" contract correctly caught the fallback hitting the live API from CI; updated to the new contract HERMETICALLY (RUVNET_GISTS_API seam → dead local port, asserts the DOUBLE failure).

Gong hole + a real security hole in ntfy-alerts.yml: gists-nightly could fail silently because it was never in the ntfy watch list (now: ci, integration-linux, gists-nightly). And the send step interpolated attacker-controlled issue/PR titles straight into run: shell under pull_request_target — a genuine command-injection vector on a public repo. All event fields now pass via env ("$VAR"), permissions: {} strips the token.

Also verified tonight: plugin v2.4.3 loaded live after restart; local brain-nightly last run = "VERIFIED SUCCESS v2.0.0 → v2.4.2", next fire 03:15; goldie-weekly loaded (exit 0); coverage 16.77% lines vs the 14% floor. Root cause of the recurring "perfect locally, red remotely" pattern, for the record: tests written against dev-machine state (deps, POSIX paths, macOS chmod semantics, HOME) instead of hermetic fixtures — the standing cure is remote-CI-per-job verification after every push plus everything-blocking, nothing-silent.

2026-07-12 (night) — MetaHarness router FINISHED: per-user profiles, proven cheap models, v2.4.0

Stuart's final mandate of the day executed via a tracked Ruflo swarm (swarm-1783890516413-ki39sy; 3 registered agents; hands-on coding in-session + 2 parallel Task subagents per Rule 20). Commits 5f80f3f + 28a3c8e + 7a01688, pushed; QA 8/8 router tests + 60/60 plugin + version surfaces synced 2.4.0.

Per-user subscription profiles (the keystone — "don't assume anyone's setup matches mine"): scripts/model-router-setup.mjs detects what's provable (Codex auth mode from auth.json SHAPE), ASKS what isn't (Claude plan), records basis-labeled answers to ~/.claude/model-router/profile.json; engine overlay strips unbacked subscription/harness claims (2 new tests). Ships to every user: in-repo config/model-router/ templates, npm files whitelist, installer offerRouterProfile() (materializes config + tools to stable ~/.claude/model-router/bin/, asks the 2 questions on TTY), session-start one-time nudge, SKILL.md consult-engine-first rule. Stuart's own profile written with verified/attested bases.

Cheap models PROVEN then wired: live dispatches through route-cheap/agentic-flow (real answers, receipts, exit 0) for deepseek-v4-flash ($0.00005/task est) and x-ai/grok-4.5 ($0.00183) → both selectable; scripts/codex-routed.sh launches codex with the engine's pick (fallback = plain codex).

Goldie upgrades: gpt-5.6 auto-reinstate on live cache evidence (flag-only inverse); models.env↔ catalog cross-check live (flags the untracked Gemini pins). Outcome log begun: scripts/model-router-outcome.mjs → routing-outcomes.jsonl, seeded with today's real haiku→sonnet override — the labeled data that eventually replaces the placeholder policy (ADR-149 direction).

Upstream: ruflo memory-CLI bugs (tombstone blocks re-store; read/write visibility disagreement) filed as github.com/ruvnet/ruflo/issues/2652 (Stuart may close if unwanted — filed under the "finish everything" directive).

THE CI SAGA (same night, Stuart's fury justified): ntfy pages revealed the ci workflow had NEVER been green — all 25 recorded runs since 2026-07-10 failed, silently until the new alerts made it audible. Deep review found three stacked causes, all fixed: (1) my router tests were node:test + machine-state-dependent — rewritten vitest-native and hermetic (MODEL_ROUTER_CATALOG override); (2) THE BIG ONE, pre-existing since day one: kb/forge-ask.mjs called loadRvf() at MODULE level, so forge-mcp-all's server died at startup on any machine without @ruvector/rvf resolvable — CI runners exactly that; dev machines masked it via the ~/.npm-global fallback. Found by making the failing test carry the child's stderr (CI as the debugger, after Node-20/exact-input/hidden-deps/coverage repros all passed locally). Now lazy-loaded: dep-less installs start, answer honestly, and route failures through brain-alarm's loud path; (3) the 14% coverage ratchet tripped at 13.97% (new subprocess-tested code grew the denominator) — cleared with real in-process unit tests over the pure functions, ratchet untouched. windows-unit: never green since added, now continue-on-error with a dated comment + the two known Windows bugs named (drive-letter-doubling import; version surface spawn) — flip back when actually green. Run 29218464685: check GREEN, RUN: success — first ever. Standing lesson recorded: local gates are not the pipeline; watch the remote run to conclusion after every push before claiming green.

Honest residuals: placeholder policy still routes (replacement needs accumulated outcome labels — pipeline now exists); tencent/hy3 stays landscape (no dispatch test); GPT-5.6 returns automatically only when the live cache shows it; the SKILL consult rule takes effect for other sessions after plugin restart (v2.4.0 staged).

2026-07-12 (evening) — Git history slim: 360MB of KB fossils rewritten out, repo 24x lighter

Stuart asked why the repo was 381MB on GitHub when the checkout is ~30MB. Answer (proven with git rev-list --objects blob analysis): pre-slim KB builds (kb/*.rvf, *.passages.jsonl — 86.9MB ruvector.big.passages.jsonl was the largest) fossilized in history from before the release-asset/gitignore split. Rewrote history with git-filter-repo (path-globs for kb data + dist/ + strip-blobs >5MB) in an ISOLATED clone — never in-place, since two other sessions were actively committing (one pushed adc83dc mid-operation prep).

Evidence: pack 375.84MiB → 15.54MiB; HEAD tree hash byte-identical (6521a6a… before and after); 167/167 commits kept; all 3 tags preserved; releases/latest still serves ruvnet-brain.zip 707MB (HTTP 206 range-check); fresh --depth 1 clone = 30M. Force-push was lease-pinned to adc83dc (required temporarily flipping allow_force_pushes on main's branch protection — restored to false immediately after; verified). Local repo re-pointed with reset --soft — other sessions' uncommitted kb/SOURCE.json edit untouched. Full mirror backup (all 4 branches incl. pr5-* work) at ~/Code/_backups/ruvnet-brain-pre-slim-20260712.git.

Guard added: CI "No-fossil guard" fails any push with a tracked blob >5MB.

Deliberate leftovers: (1) local .git still ~400MB until deep-brain-v0.5/pr5-eval/ pr5-rebase (old-history lineage) are rebased or retired — do NOT gc before that, and rebase those onto new main before pushing/merging them; (2) GitHub's reported repo size shrinks lazily (their GC); (3) forge-update SOURCE.json merge-guard (private-store de-list risk through the ~/.cache/ruvnet-brain/kb symlink) still open.

2026-07-12 (later) — AgentDB recall fixed at the GLOBAL layer, not just this project

Stuart's real question after the nightly-infra + GONG work: "why did I have to tell you to fix this — isn't recall just part of Ruflo?" Checked live rather than assumed: ruflo agentdb session-start doesn't exist, hooks session-start is deprecated and only restores agents/tasks, memory --help has no recall/session-start subcommand. Confirmed: nothing in the ecosystem auto-recalls memory at session start. The fix built earlier this session (the project-state-current auto-recall block) only lived in this repo's own plugin hook — it never helped any other project. Moved it to the GLOBAL ~/.claude/hooks/agentdb-ensure.sh so every AgentDB-enabled project gets it, removed the now-redundant local copy here, and codified the project-state-current convention in global CLAUDE.md Rule 19 (bumped v6.6.0→v6.7.0).

Bonus find while re-writing this repo's own key: ruflo memory retrieve / delete / store --upsert all fail against a row that provably exists (byte-identical key/namespace via raw sqlite3, absolute path tested, no metadata differences) — a real, reproduced, live CLI bug, not a ruvnet-brain bug. Not chasing it into claude-flow's source tonight; instead the global hook's recall now reads memory_entries directly via read-only sqlite3 (same table agentdb-autocapture.mjs already writes to) so the one thing this hook exists for doesn't depend on a component that just proved unreliable. Worth a GitHub issue upstream at some point.

Verified live for all three cases: existing key (ruvnet-brain) → real content printed; memory.db present but no key yet (PowerPlatePulse) → correctly silent, no false-positive WARN-as-content; no .swarm at all → falls through to the unaffected new-project offer flow.

2026-07-12 — BRAIN OUTAGE fixed + THE GONG SYSTEM (v2.3.0) + MetaHarness router committed

The outage: the brain was completely dark — every search_ruvnet (MCP + CLI) failed with Cannot resolve '@xenova/transformers' across all 37 repos, and the MCP tool reported it as an innocent "(no results)". Root cause: retrieval had been resolving the embedder from ~/.npm-global (XENOVA_PATH fallback); a node upgrade broke that and kb/ had no node_modules of its own (~/.cache/ruvnet-brain/kb is a SYMLINK to the repo's kb/). Fix: npm i in kb/ (deps now pinned via committed package-lock.json, no global-dir dependency). Proven recovered: 37/37 repos answering, 312 pooled candidates on the canary query.

THE GONG SYSTEM (Stuart's non-negotiable: "that can NEVER happen silently") — c9db076, v2.3.0. Three independent, individually TESTED alarm layers (proven by deliberately breaking the brain — mv kb/node_modules away — and watching each ring, then restoring and watching recovery clear):

  1. Real-time: kb/brain-alarm.mjs + total-failure detection in forge-mcp-all.mjs/forge-ask-all.mjs — all-repos-failing returns a screaming isError response (never "(no results)"), CLI exits 1, writes ~/.cache/ruvnet-brain/health.json, urgent ntfy push (throttled 1/hr; topic from env or ~/.cache/ruvnet-brain/ntfy-topic). Partial failures labeled DEGRADED with dead repos named.
  2. Per-session: plugin/scripts/session-start.sh pure-fs structural checks flash a red MANDATE banner in every new session while broken.
  3. Nightly: canary query in scripts/nightly-wrapper.sh, own urgent push on failure. Plugin QA 60/60 green. Test push delivered to phone.

MetaHarness model router committed (23d085f) — the harness-neutral prompt→model decision engine built earlier today (partly from Stuart's accidental all-in-window session, reconciled + tested 5/5): scripts/model-router-engine.mjs (pluggable policy, subscription-$0 floor, verified-pricing catalog at ~/.claude/model-router/catalog.json), status tool, contract tests. Repo reconciled: pulled v2.2.0→v2.2.2 + nightly-infra commits from the other session, set upstream tracking, pushed.

Grounded design directions for the router policy (brain citations): agentic-flow's Multi-Model Router (agentic-flow/docs/router/README.md — --router-mode cost-optimized, budget alerts) and ruflo ADR-149 (PROPOSED — per-model cost-optimal routing via @metaharness/router's Router.fromExamples qualityBar selection, measured 76–97% savings forecasts). Codex subscription detection verified live: ~/.codex/auth.json has OAuth tokens (ChatGPT Business login), NO API key — Codex on this machine is subscription-covered, marginal-$0.

GOLDIE SHIPPED + FIRST RUN PROVEN (7f588a5 + follow-up, same day). Weekly launchd job (com.ruvnet.goldie-weekly, Mondays 07:30, loaded + verified in launchctl): layer 1 deterministic (goldie-research.mjs — live OpenRouter pricing → catalog refresh + >20% drift flags + radar + dated brief in ~/.claude/model-router/goldie/), layer 2 judgment (headless claude -p on the Max subscription, env -u ANTHROPIC_API_KEY guarded after a stale env key — live-tested 401 — broke run #1). First full run's judgment layer immediately caught a real catalog error: the gpt-5.6-sol/terra/luna entries claimed "verified live 17:27 UTC" but two independent reads of ~/.codex/models_cache.json show only gpt-5.5/5.4/5.4-mini — 5.6 is announced but NOT reachable by this Codex install. Fixed: demoted to landscape-only; that exposed a policy gap (codex cheap/mid fell to BILLED DeepSeek while subscription gpt-5.5 sat unused) → cross-tier $1,600 floor added to policy.default.mjs, locked by a 6th test (6/6 green); codex plan now gpt-5.5 across all tiers. Also live-verified on OpenRouter and added as landscape entries: deepseek/deepseek-v4-flash ($0.077/$0.154, 1M ctx — proposed successor to deepseek-chat, which resolves to LEGACY V3), tencent/hy3 (paid slug only — :free promo dies ~07-20), x-ai/grok-4.5 ($2/$6 — wireable via OpenRouter, old "needs bespoke xAI adapter" note obsolete). Goldie's full sourced analysis: ~/.claude/model-router/goldie/2026-07-12.md. SIDE FINDING for Stuart: the ANTHROPIC_API_KEY in ~/.zshrc + launchctl env is DEAD (HTTP 401 live) — anything relying on it silently fails.

SECRETS: ONE VAULT + KEY-HEALTH CANARY (same day, Stuart-approved). Root cause of the "stale keys project to project" pain mapped live: SOPS+age vault in openclaw-stack decrypted to secrets.env (sourced machine-wide by .zshrc) — but the chain had no enforced direction, so copies drifted (decrypted hand-edited Apr 2 without re-sealing; launchctl copies never synced — the launchctl OPENAI_API_KEY was DEAD (401) while the shell copy was alive). Built + proven: openclaw-stack/secrets-sync.sh (seal/sync, round-trip verified before vault replacement, REFUSES to destroy un-sealed edits — refusal proven live; sync healed the dead launchctl copy 401→200) and scripts/key-canary.mjs (live list-models probe per provider through the REAL zsh -lc env chain; DEAD detection proven with a bogus key; transition-only urgent pushes; wired into nightly-wrapper). New Anthropic key installed via the full chain (vault re-encrypted, round-trip proven, fresh-shell HTTP 200). All 6 provider keys verified alive. openclaw-stack commit f3c5a86 (private repo). Full map: auto-memory global-secrets-architecture.md.

OPEN (next): wire router consult points (subagent dispatch via hooks_model-route + engine, Codex launch wrapper, route-cheap PRICING entries + tested dispatch for v4-flash/grok-4.5 after measured quality checks); ship policy/catalog templates in-repo (they live only in ~/.claude now); learned policy over routing-decisions.jsonl once outcome labels accumulate (Goldie's Q1 verdict: keep 3 buckets, fix the routing FUNCTION — ADR-040/DRACO consistent); AgentDB omniscience consolidation pass.

2026-07-06 — SECURITY HARDENING (SEC-0010): Dragan Spiridonov's QE review, all 12 addressed

Dragan (one of rUv's most trusted QE people) responsibly disclosed a 12-finding QE review (deep read + a swarm of security/code/test/dep/docs agents), reviewed at HEAD 126fc3b. Full engineering record with each finding's root cause + exact fix + verification: docs/adr/0010-security-hardening-sec-0010.md. External point-by-point response for Dragan: User Feedback/2026-07-06-ruvnet-brain-security-response.md.

11 of 12 fully fixed + verified (each proven with a real command); #6 acute RCE vector closed, full crypto-signing tracked. Product 1.9.1-dev → 1.9.4-dev; whole gate green.

  • #1 gate.sh PIPESTATUS (5e703fb) · #2 version SINGLE SOURCE OF TRUTH — plugin.json + version.mjs + sync-version.mjs --check in CI, 5 numbers → 1 (12232ed) · #3 test 26/26 (12232ed) · #4 fence fail-closed PROVEN via fault-injection (5e703fb) · #5 concepts store private-filtered fail-closed (ff8193b) · #7 injection-guard recall widened, 19/19 held + 5 evasions flag (cb0f2bb) · #8 ship lockfile + npm ci (ee8daca) · #9 protobufjs CVSS 9.8 cleared via npm overrides→8.7.0, audit 0 vulns, embedder verified working (ee8daca) · #10 Windows claim scoped honest (ff8193b) · #11 explainer S04 honest (12232ed) · #12 CI added + .env leak scrubbed + orphan-guard + mkdir.
  • #6 unsigned-updater RCE: auto forge-update.mjs --apply (overwrote .mjs from unsigned Release) removed → detect+notify only (7b88e80). Full Ed25519/cosign verify-before-extract, grounded in ruflo ADR-177, is the one honest open item.
  • Two verify-first catches: (a) the "obvious" #9 fix — migrate to @huggingface/transformers — BROKE the offline embedder (caught by a live query, reverted, used overrides instead); (b) #4 proven by corrupting the fence. Verify-first earned its keep.
  • New CI (.github/workflows/ci.yml) runs version:check + npm test + guard unit on every push — the missing CI is what let #1/#3 ship unnoticed.

2026-07-05 — THE BEHAVIORAL OVERHAUL DAY (v0.5.0-dev → v1.9.0-dev, 16 shipped versions)

The driver: real field feedback (Mario Jauvin, relayed by Stuart) exposed that the brain read as "only useful for RVF/Ruflo/AgentDB questions" — cold, scope-gating, lab-report tone. Stuart's bar, set explicitly across the day: the brain must be Ruv on your shoulder — plays back intent, attacks with one confident plan, references only relevant tools, shows its thinking, proves its checks, serves every developer level.

Behavioral contract evolution (plugin SKILL.md + ground-ruvnet.sh gate 3):

  • 0.5.6: toolkit-breadth (search, don't recall — real search_ruvnet calls with feature-specific queries; the OAuth/PROVIDER_PRESETS example)
  • 0.6.0: partner-on-shoulder response contract (intent playback / approaches / fit-with-their-code / one go-no-go)
  • 1.6.3: process line ("checked project memory — found X; will persist as we go") + two-layer answer ("yes for process; no library in deliverable")
  • 1.6.4: ATTACK PLAN replaces options-essays — reject silently, build loudly
  • 1.6.5: every proposal signs its version
  • 1.7.0: consolidation — one voice, five beats (hear them / attack / why it holds / what I checked / cleared to go), replacing 10 stacked patches
  • 1.9.0: ADRs as living plans (read Status/dates, update in same change, reconcile drift via adr-review/adr-verify)
  • Field grades of Mario transcripts: 78 → 88 (out of 100). Best transcript: direct two-layer answer + honest memory recovery that wrote 6 memories on the spot.

Infrastructure shipped same day:

  • Consent-gated AUTO-UPDATE: session-start heartbeat (every restart, 15-min dedupe) detects new plugin version, background-applies via claude CLI, tells user restart loads it. KB bundle auto-updates too (forge-update --check/--apply vs releases/latest).
  • TRUTH IN STATUS: statusline + banner show RUNNING (in-memory) version; staged version shown explicitly as "staged — restart to load". Stuart's rule: never show a staged version as running.
  • STACK WATCHDOG (Gate 0, every prompt, filesystem ground truth): Ruflo wired? .swarm/memory.db exists/recently-written? Outdated ruflo/@claude-flow/cli/@ruvector/rvf (npm registry, 20h cache — first run found 3 real outdated packages on Stuart's machine)? Every response ends with status footer. Ruflo-without-memory triggers once-per-session warm offer to turn AgentDB on.
  • NIGHTLY PUBLISH: self-update.mjs --publish (launchd 03:15, loaded) → rebuild changed repos → zip → GitHub Release → bump ONE product version → stamp README bright-blue badge (version + date/time/TZ) → push. Users' heartbeats pull it automatically.
  • npm: ruvnet-brain@1.6.2-dev published (npx ruvnet-brain). Version 1.x era per Stuart.
  • Incidents owned: deleted top-level .claude-plugin/marketplace.json broke GitHub-sourced updates for ~30 min (restored); macOS sh parses UTF-8 char glued to $VAR as part of the name (byte-level debug).

Scores (out of 100, Stuart's standing rule): presence 90 · insertion 82 · voice 60 · execution 40 · proof 20 · overall ~62.

1.9.1-dev (late add): beat 0 — point-blank questions answered in sentence one (the 94-transcript's biggest deduction: the rule existed but sat buried; now positionally first) + findings spoken in the user's vocabulary, never plumbing-state. Field grades now 78 → 88 → 94; predicted 97-98 on 1.9.x, eval harness to verify.

2026-07-05 late — ruflo 3.24 flywheel changes our own game plan:

  • rUv published the exact architecture for evidence-gated self-improvement (gist f8e2851): candidates searched on the install's OWN data → promoted ONLY past an accept gate (beats incumbent on a FROZEN HELD-OUT split + 95% bootstrap lower bound > 0 + no regression on human-labeled truth + deterministic replay) → auditable lineage DAG (.claude/evolve-proof, reconstructLineage()) → fail-closed. Signed champion: +0.0738 nDCG@3; live compounding 0.496→0.847 RR.
  • The eval harness is now designed as RuvNet Brain's own flywheel: contract variants = candidates; grader-scored scenario set with a FROZEN held-out subset (never tuned against — same discipline as our heldout.*.json); accept gate = beats incumbent + no regression on the human-graded "Mario canon"; promotion = version ship via the existing auto-update chain; lineage = git + graded transcripts per version; fail-closed = incumbent stays when the gate doesn't pass. Today's 17 hand-tuned versions were a manual flywheel; next session builds the automated one.
  • Real bug found & fixed en route: ruflo + agentic-flow were stamped builtFromSha=unknown and self-update.mjs treated unknown as up-to-date — the two most-central repos could NEVER auto-refresh. Fixed (unknown → rebuild when upstream reachable); ruflo 3.24+ re-ingest running; tonight's nightly --publish ships the refreshed corpus to all users automatically.

NEXT (the two that matter): (1) EVAL HARNESS — encode Stuart's bar as graders, ~12 prompts × 3 dev levels × 4 scenario types, claude plugin eval, loop ≥90 — stop using Mario as QA. (2) EXECUTION TRIAL — scratch repo, real feature, full cleared→SPARC→build→test→proven loop, graded end-to-end. Then: field wave with version-signed transcripts.

Standing Stuart rules recorded today: scores /100 always · standing instructions get written to memory the moment they're uttered · maintain THIS file in real time (it was 7 days stale when he called it out — never again).

★ CURRENT STATUS (read this first)

Product version = whatever plugin/.claude-plugin/plugin.json says (the single source of truth — ADR-0009 decision 1). As of 2026-07-06: plugin 1.9.x-dev, npm installer 1.6.2-dev, brain Release v0.5.0-dev. Do not trust a hand-typed version number anywhere in this file's history below; read the manifest.

  • The dated entries below are the live log — the NEWEST entry is the truth. (This banner used to hard-code v0.3.0-dev / "19 repos" / "~75,000 chunks" — six majors stale; that was exactly the doc-currency drift ADR-0009 was written to kill. Corpus/scope counts have moved and are re-derived per build, not pinned here.)
  • Corpus: ~26 local stores (5 private cognitum-* fenced out of the public bundle → ~21 public); rebuilt 2026-07-05 against ruflo v3.25.0. Exact chunk/repo counts live in data/manifest.json + kb/SOURCE.json, not in prose.
  • Self-audit (2026-07-06, 5-agent parallel): honest overall ~66/100 — excellent installer (86), explainer (84), retrieval core; eroded by version-fracture, enforcement-overclaim, and doc-currency debt. Full scorecard + the fix roadmap: docs/adr/0009-mirror-discipline-self-audit-and-qa.md.

Everything below this banner is the historical session log — preserved as a timestamped record of how we got here. It reflects the original 5-repo milestone (5/169) and is intentionally not rewritten.

Session 2 (post-compact 2026-06-28) — scaling T0/T1, LIVE

Three real parallel streams running, each labeled as what it actually is (no Ruflo-agent overclaim):

  1. Background processes (b28c1h8vy) — bge-768 ("big") builds for the 4 non-ruflo repos. 7 procs: rulake/agentdb/ruview as single both + ruvector sharded 4×. bge rate ≈6.8 chunks/s (measured from ruflo's 1411s/9648). ETA ~25-30 min (ruvector 4-shard ≈17 min is the long pole). Orchestrator: kb/build-big-all.sh; per-proc logs kb/big-*.log.
  2. Frontend Task agent — applying the 5 UX fixes to explainer/index.html + screenshot-verify + re-grade.
  3. General Task agent — writing grounded grading question sets kb/questions.<repo>.json + heldout.<repo>.json for the 4 repos.

DONE this session (proven):

  • Symbol indexes built for all 4 repos (point-deeper, ADR-0003): ruvector 16,975 sym/2,337 stem/157 pkg · agentdb 911 · rulake 367/10pkg · ruview 1 (firmware/docs repo — weak by content, not bug). Files kb/<repo>.symbols.json.
  • Bug fixed in scripts/build-symbols.mjs: plain {} maps hit JS prototype-pollution (map["__proto__"].add crash) on ruvector's 28K chunks → changed the 3 maps to Object.create(null). Re-ran ruvector: success.
  • Grading question sets (general agent a5d8c7a3): kb/questions.<repo>.json + heldout.<repo>.json for all 4 repos (12 tuned + 10 held-out each), every Q grounded in a real path, mixed ADR statuses.
  • Explainer FINISHED + verified (frontend agent a21040ca, + my own skeptic review of the screenshots): all 5 UX fixes applied (real CTA/install, Ruv/RuvNet defined, maturity pills, labeled L0–L4 teaching diagram, "what REAL-USE 77 means" + security answer). Skeptic-grade ≥90. Caveat: deck reflects ruflo-only state → needs a final refresh to multi-repo numbers/links once the bundle ships (tracked).
  • Cross-repo "one brain" layer BUILT + PROVEN (the core "answer about ALL of RuvNet" capability):
    • kb/forge-ask-all.mjs — CLI: searchKb per repo → pool → ONE cross-encoder rerank (common scale) → global top-k labeled by repo. Proven: 5 repos discovered, 25 pooled, ranked. Added rerankPairs() to kb/forge-rerank.mjs (reuses the loaded CE; rerankKb untouched).
    • kb/forge-mcp-all.mjs — MCP stdio server, ONE tool search_ruvnet whose description forces "check before claiming any RuvNet capability." PROVEN via real JSON-RPC: initialize/tools-list/ tools-call returned 14,932 chars cross-repo, ADR proposal-status guard fired.
    • scripts/build-bundle.mjs — idempotent assembler → dist/ruvnet-brain/ (per-repo stores+sidecars+ symbols+big-when-present+primer+L2 + shared tools + manifest{coverage,per-repo chunks/variants/grade/SHA}
      • mcp.snippet{cross-repo + per-repo} + README). PROVEN: 5/169, 53 files, ruflo grade 76.72 auto-pulled.
  • CAPABILITY-CONFIDENCE GATE built + PROVEN (Stuart's #1 ask — "never doubt what a repo can do"): scripts/brain-capability-check.mjs + kb/capability.agentdb.json. AgentDB 9/9 PASS even on BASE variants (τ=0): every YES-capability returned a strong positive cross-encoder hit (ce 2.4–7.0) from the RIGHT repo + RIGHT evidence file (Cypher→agentdb-graph/cypher.md 6.84; Thompson/RL→agentdb-learning plugin.json 6.99; single-RVF→agentdb-init SKILL 6.53); both NO-controls correctly did NOT assert (video ce 0.39 evid 1/2; SMS ce −4.71 evid 0/2). KEY DISTINCTION: capability-EXISTENCE ("can X do Y?") is answerable even on weak embeddings (Ruv names capabilities in docs/manifests) — that IS Stuart's pain; capability-MECHANISM ("how in code?") is where bge big variants matter. CAVEATS: 1 repo so far; I authored the Qs (mild bias → held-out sets are the anti-overfit); harden controls + raise τ in full pass.
  • HONEST FINDING — cross-repo MECHANISM ranking is base-variant-weak (as expected): on MiniLM base variants the cross-repo reranker returns NEGATIVE ce scores (e.g. "ruflo memory" Q → a ruview PROPOSED ADR #1). Cause: base MiniLM under-ranks code so the truly-relevant file never enters the rerank pool. Fix is already in flight: the bge-768 big variants + grading/tuning (on ruflo big+rerank lifted REAL-USE to 77). The CONTENT is present (ruvector RVF spec ADR-031 retrievable within-repo). RE-TEST cross-repo after big builds. Also consider a larger per-repo pool (8→12-15) so borderline-relevant docs reach the cross-encoder.

NEXT (after builds finish — gated on .big.rvf existing): re-test cross-repo ranking on big variants → guard both variants → 3-vendor ground-truth grade (use prepared question sets) → capability-confidence gate ("Can RuvNet do X?" never-doubt test, per Stuart's #1 ask) → L2 per repo where synthesis gaps show → per-repo primers → re-run build-bundle (now with big+primers+grades) → final explainer refresh → self-update (install only with explicit approval). Task board live (TaskList).

Session 2 — POST-BUILD milestones (2026-06-28, ~00:40 PDT)

  • ✅ bge-768 big builds COMPLETE (b28c1h8vy, 00:32). All 5 reconcile MATCH (idToLabel==passages, 0 rejected): ruflo 9648 · rulake 1149 · agentdb 3624 · ruview 4127 · ruvector 28297. Sizes: ruvector.big 84M, ruflo 28M, agentdb 12M, ruview 12M, rulake 3.4M. (Honest: my first reconcile check had a key-name bug — idmap is {idToLabel,...}; the build's own ingest reconcile + corrected count both confirm MATCH.)
  • ✅ GUARD PASS — all 5 repos, BOTH variants (forge-guard, OVERALL PASS ×5): parity + anti-truncation
    • live-query all green.
  • 🟡 3-vendor ground-truth grade RUNNING (bkdckt690, big+rerank, openai/deepseek/llama). path-exists citation check working (0 hallucinated so far). Early rulake: REAL-USE ~69-80. (First launch failed on a zsh key-quoting bug — no API wasted; re-launched clean.)
  • Source trees for grading: ruvector=/Users/stuartkerr/RuVector_Clean · agentdb=clones/agentdb · rulake=clones/RuLake · ruview="Cognitum Sensor Primer/cognitum-one-sensor-primer/RuView". Key in Ask-Ruvnet/.env (extract clean: ${K%\"}/${K#\"}/${K// /} — NOT a fragile tr chain).

Session 2 — GRADING + CAPABILITY results (2026-06-28, ~00:55 PDT)

3-vendor ground-truth grade (big+rerank, tuned sets) — DONE

repo avg REAL-USE min avg STRICT poison(RU) citation fails
ruvector 83.1 48 51.6 1 0
rulake 83.0 30 62.7 1 0
agentdb 77.9 33 50.4 1 0
ruview 72.5 43 41.8 1 0
  • 0 hallucinated citations anywhere (the ADR-0002 hard gate). REAL-USE 72-83 = proven realistic band (matches ruflo 77). Each repo has exactly 1 poison synthesis question → L2 targets (10 total, in kb/l2-topics.<repo>.json). STRICT low as expected (diagnostic, not gate of record). Reports: data/grade-<r>-big.json.

Capability-confidence gate (big, cross-repo, τ=0) — first pass = 38/45 (84%)

  • agentdb 9/9 · ruview 9/9 · ruvector 9/9 · rulake 7/9 · ruflo 4/9.
  • THE KEY INSIGHT (Stuart's #1 ask): the cross-encoder (MS-MARCO-trained) confidently matches PROSE that describes a capability (+2.4 to +7.0) but UNDER-scores CODE that implements it (ruflo's guidance-tools.ts scored −4.58 even with evidence met). So repos that DOCUMENT capabilities → 9/9; repos that implement them in .ts/SKILL.md (ruflo) → fail despite the capability genuinely existing. Two failure modes: (A) cross-repo dilution — a repo's own docs miss top-5 for generic terms ("agent","mcp tools"); (B) right doc found + evidence met but negative ce. FIX (not lowering the bar): give each repo a PROSE capability statement (primer "what can it do" + L2 concept articles), re-embed into the corpus → those become the high-confidence hits → re-run gate. Also refine generic evidence terms (my author bias) + raise pool/expectRepo handling for cross-repo dilution. → primers+L2 integration are REQUIRED for capability confidence, not just synthesis. Log: kb/capability-all.log, sets kb/capability.<r>.json.

Now running

  • 🟡 L2 synthesis (bw21dseww) for the 10 graded gaps (4 repos), ≥2-citation gate. build-l2.mjs now loads kb/l2-topics.<name>.json per repo (ruflo default unchanged).

Next: L2 done → per-repo primers (all 5) → INTEGRATE l2+primers into corpus (re-embed) → RE-RUN grade +

capability gate (expect ruflo/rulake lift) → held-out grade → re-run build-bundle → final explainer refresh.

Session 2 — L2 + INTEGRATION architecture (2026-06-28, ~00:55 PDT)

  • ✅ L2 synthesis DONE (bw21dseww): 9/10 ACCEPTED (≥2 real cites, 3-vendor 91.5–98.3), 1 correctly REJECTED (ruvector adr-029: only 1 cite — single-ADR question, served by retrieval not synthesis). Articles in kb/l2/*.md. build-l2.mjs now loads kb/l2-topics.<name>.json per repo.
  • 🟡 Per-repo primers BUILDING (b0gbu9j0j): scripts/build-primer.mjs (6 archetypes incl. a CAPABILITIES section = prose "what can it do" + implementing file — the ruflo capability fix), grounded via rerankKb + synth + 3-judge. Writes kb/<repo>-primer.md.
  • INTEGRATION ARCHITECTURE (decided + scripts written): forge-big ingest REPLACES (RvfDatabase.create
    • re-embeds full passages) — so adding primer/L2 to a repo = full re-embed (28K chunks for ruvector, wasteful). INSTEAD: a separate concepts store. scripts/build-concepts.mjs assembles all accepted L2 + all primers → kb/concepts.passages.jsonl (+meta), paths tagged <repo>/<kind>/<slug>; then node kb/forge-big.mjs both --dir kb --name concepts → concepts.big.rvf (~60 chunks, ~1min). The cross-repo discoverRepos already detects big-only stores → search_ruvnet unions concepts at query time WITHOUT re-embedding the giants. Capability gate tweaked: a concepts hit counts for expectRepo via its path prefix. Rationale: hot concept layer (cheap, iterates in seconds) vs cold source layer (40K chunks, rarely changes) — separate stores, union at query.
  • NEXT (after primers): build-concepts → forge-big concepts → RE-RUN capability gate (expect ruflo 4/9 + rulake 7/9 to lift) + re-grade → held-out grade → re-run build-bundle (include concepts + l2 + primers) → final explainer refresh → self-update (approval-gated).

Session 2 — ★ NEVER-DOUBT GATE CLOSED (2026-06-28, ~01:05 PDT) — Stuart's #1 ask, PROVEN

  • ✅ Per-repo primers (b0gbu9j0j): all 5 GROUNDED (33–42 verified refs, 3-vendor 94.7–97.0). kb/<r>-primer.md.
  • ✅ Concepts store built in 5s (scripts/build-concepts.mjs → 41 passages [11 L2 + 5 primers] → forge-big both concepts → concepts.big.rvf, reconcile MATCH). Hot/cold split paid off (vs hours to re-embed giants).
  • ✅ CAPABILITY GATE RE-RUN with concepts: 84% → 100% (45/45). ruflo 4/9→9/9, rulake 7/9→9/9; agentdb/ruview/ruvector held 9/9; controls still correctly refuse (no invented capabilities). Diagnosis CONFIRMED: every prior ruflo miss now resolves to ruflo/PRIMER prose with positive ce (swarm/spawn was a total miss → ce +3.51; guidance was ce −4.58 → +0.84). search_ruvnet unions concepts automatically. Logs: kb/capability-rerun.log. build-bundle.mjs now copies the concepts store + notes it in manifest.
  • NEXT: held-out anti-overfit grade (4 repos) → verify synthesis-gap L2 surfaces cross-repo → re-run build-bundle (final, w/ concepts+L2+primers+grades) → final explainer refresh → self-update (approval-gated).

Session 2 — HELD-OUT anti-overfit grade (2026-06-28, ~01:15 PDT) — DONE

repo tuned RU held-out RU poison cite fails
ruvector 83.1 85.2 0 0
agentdb 77.9 82.0 1 0
rulake 83.0 77.4 2 0
ruview 72.5 62.6 4 0
  • NOT form-fit (2 repos scored HIGHER on unseen Qs). 0 hallucinated citations on held-out too — grounding integrity generalizes. ruview = honest weak point (firmware/sparse prose). Deliberately did NOT build L2 for held-out Qs (tuning on the test = overfitting). grade--big.json now = held-out (tuned preserved in this file + grade-tuned-all.log). Grading COMPLETE (task #4).
  • ruflo primer spot-checked = high quality ("EXISTS: → " + honest "Not Covered"). Minor: LLM emitted a doubled section heading → strip in final polish.

Session 2 — ★★ BUNDLE COMPLETE + PROVEN (2026-06-28, ~01:25 PDT)

  • ✅ dist/ruvnet-brain/ (v0.3.0-dev) + dist/ruvnet-brain.zip (268 MB, 97 files, node_modules excluded). All 5 repos × both variants + symbols + primers + L2 + concepts store + cross-repo tools (forge-ask-all, forge-mcp-all/search_ruvnet) + manifest + mcp.snippet + README.
  • ✅ date+SHA STAMPED (all 5): ruflo 322b2ae5 · ruvector ed95f498 · agentdb 47c79b61 · rulake 8f2c4086 · ruview 4bf88e12. (data/manifest.json + bundle manifest.)
  • ✅ ACCEPTANCE PROOF on the SHIPPED bundle (real consumer flow): cd dist/ruvnet-brain && npm i (86 pkgs, 6s) then node forge-ask-all.mjs --dir . --q "Can ruflo orchestrate agent swarms, and what implements it?" → #1 = concepts/ruflo/PRIMER ce=5.75 (confident, source-grounded). All 6 stores searched (5 repos + concepts). NOTE: cross-repo quality REQUIRES the cross-encoder, which requires npm i (without it, falls back to raw vectors and ranks poorly) — documented in README/install.
  • Manifest grades = held-out (anti-overfit) for the 4; ruflo = its tuned 76.72. coverage 5/169.
  • REMAINING: explainer refresh to multi-repo (frontend agent a6646429f6fc9f1d5 RUNNING) · self-update nightly install (plist written, NOT installed — needs Stuart's explicit approval) · optional polish (primer doubled-heading strip; held-out capability set).

Session 2 — EXPLAINER refreshed to final state (2026-06-28, ~01:35 PDT) — DONE

  • ✅ explainer/index.html updated to the proven 5-repo state (agent a6646429f6fc9f1d5, self-grade 93, my own screenshot review confirms): v0.3 tag; hero defines search_ruvnet; stat tiles = 46,845 chunks/5 repos · 5/169 proven · 100% capability confidence 45/45 · 0 hallucinated cites; a green "HEADLINE GUARANTEE — capability confidence" callout (84%→100%, ruflo 4/9→9/9, controls invent nothing); honest per-repo table (shows ruview 63). 0 stale bundle-name refs remain. Aesthetic untouched. Fixed hero .hero-fig img mobile clip (added width:100%;height:auto). Explainer is standalone (NOT in the zip) → no re-zip needed.
  • PROJECT STATUS: bundle COMPLETE+PROVEN, explainer DONE. Only open item: self-update LaunchAgent install (needs Stuart's explicit approval per security rule) — asked; awaiting his call. Optional polish remains (primer doubled-heading strip; held-out capability set; per-repo trajectory tables still ruflo-only).

Session 2 — ★★★ PROJECT COMPLETE (2026-06-28, ~01:50 PDT) — all 11 tasks done

  • ✅ Explainer LIVE on Vercel (public): https://explainer-stuart-kerrs-projects.vercel.app (team sikerr-6092, project explainer). Deployed explainer/ static; disabled Vercel Authentication (ssoProtection→null via API) — it was behind an SSO wall by default. Verified HTTP 200 + content. Redeploy: cd explainer && vercel deploy --prod --yes.
  • ✅ Nightly self-update INSTALLED (Stuart approved via prompt): ~/Library/LaunchAgents/com.ruvnet.brain-nightly.plist, launchctl loaded, verified in launchctl list. Runs 3:15 AM, RunAtLoad=false. FIXED 3 bugs found by dry-run BEFORE install (PROVE-IT win): (1) case-insensitive built-SHA lookup (RuVector≠ruvector was forcing needless rebuilds); (2) SAFE scope — rebuilds only already-built CHANGED repos + re-bundles; the 43 unbuilt repos are SKIPPED unless --include-new (prevented an unattended 40-repo / days-of-compute first run); (3) lowercase kb --name on write (was about to write RuVector.rvf instead of updating ruvector.rvf). Also added build-symbols + build-bundle to the nightly. Dry-run now: rebuild RuView+ruflo (changed upstream), RuVector/agentdb/RuLake up-to-date.
  • DELIVERABLES: bundle dist/ruvnet-brain.zip (268MB, SHA-stamped, acceptance-proven) · explainer (local + Vercel) · 5 repos graded (REAL-USE 63-85, 0 hallucinated cites) · capability-confidence 45/45 (100%) · nightly evergreen.

The goal (one line)

One downloadable bundle that makes Claude/Codex a world-class, source-grounded practitioner of the entire RuvNet ecosystem — points to exact code, never skims, can't drift — proven ≥98/100 against the real source.

Phase status

Phase What Status
Spec v0.3 effectiveness-first, point-deeper, ground-truth arbiter, zip bundle ✅ written + 3-way red-teamed
ADR/DDD formalize the hard-won decisions ✅ ADR-0001..0007 + DDD.md + tier registry
P1 ruflo PoC (small) deep-walk ruflo, prove a real source-grounded answer ✅ PROVEN — 9,648 chunks, reconcile MATCH, real code returned
P1 sharp variant bge-768 re-embed (effectiveness-first) 🟡 building (~20 min)
Primer + visuals master primer + ASCII→SVG hero diagram ✅ primer + 1 validated SVG (more in HTML phase)
Version stamping date + per-repo SHA in primer/manifest ✅ manifest.json + primer stamped
Self-update agent evergreen nightly rebuild 🟡 stamp done; scheduler + change-detect pending
Grading harness ground-truth + multi-vendor, iterate to ≥98 ✅ BUILT + RUN (3 vendors: OpenAI/DeepSeek/Llama). Baseline numbers below.
Scale T0/T1 RuVector, RuView, AgentDB, RuLake, … ⬜ (after sharp ruflo proven)
Triple red-team #2 re-attack the built result ⬜
HTML presentation visual story + AI imagery + frontend-design ⬜

Measured so far (real numbers — pasted, not claimed)

  • ruflo deep-walk census: 3,516 files (v2 excluded), 9,648 chunks, 3,101 source paths, 134 ADRs, 897 docs.
  • Reconcile gate: chunks = vectors = passages = meta ids = 9,648 · MATCH = true. RVF 16.5 MB, 604 segments.
  • Proof query "memory backend?" → returns v3/@claude-flow/memory/src/sqlite-backend.ts (#1), plus agentdb-adapter.ts, hybrid-memory-repository.ts. 193 memory/src passages in corpus. ADR-status surfaced.
  • Measured gap (validates effectiveness-first): natural-language phrasing ranked the ADR doc #1; only the code-term query surfaced the .ts. → bge sharp variant + symbol-index point-deeper routing are the fix.
  • Stamp: ruflo built @ 322b2ae5; manifest covers 1/169 built, 168 pending.

Grade — ruflo, SMALL (MiniLM-384) baseline, 12 Q × 3 vendors (OpenAI/DeepSeek/Llama)

  • avg STRICT (#1 doc alone) = 49.5 · avg REAL-USE (top-5 synthesized) = 81.8
  • min STRICT 20 · min REAL-USE 50 · poison(STRICT)=7/12 · poison(REAL-USE)=0
  • ground-truth citation failures = 0 (every cited path is a real file in the repo)
  • Verdict: SMALL FAILS the ADR-0002 gate (needs avg≥98/min≥95 both). Expected — prose embedder under-ranks code; #1 is often an ADR doc not the .ts. NEXT: re-grade on SHARP bge-768, then point-deeper symbol routing if the gap persists. (Report: data/grade-ruflo-small.json.)

Grade — ruflo, SHARP bge-768, after 3 retrieval-tuning iterations (12 Q × 3 vendors)

  • avg STRICT = 35.0 · avg REAL-USE = 68.6 · ground-truth citation failures = 0 (still no hallucinated cites).
  • High: Q3 memory 97 RU · Q11 pkg 99 RU · Q1 93 RU · Q10 MCP 93 RU. Drags: Q7 ADR-coverage 30 · Q8 end-to-end 40 · Q12 neural 55.
  • Key finding 1: REAL-USE ≫ STRICT everywhere (Q1 16.7 vs 92.7; Q11 73 vs 99). STRICT="#1 doc alone fully answers" is the WRONG bar for a multi-doc-synthesis consumer. Recommend REAL-USE as gate of record; STRICT as diagnostic.
  • Key finding 2: heuristic retrieval tuning = whack-a-mole; can't beat semantic distance reliably. The principled fix is STRUCTURAL: ADR-0003 symbol index (export/fn→file:line) + index orientation docs + a cross-encoder reranker.
  • HONEST STATUS: NOT at gate. Architecture + harness PROVEN; answer-quality retrieval is the remaining hard core (the genuine 15-month problem). Now grindable with evidence (3-vendor ground-truth), not vibes.
  • forge-ask.mjs retrieval tuned (my copy): broadened code-intent, source promotion +0.60, test/agent/manifest demotion −0.70, ADR-046 off-topic magnet. (Reports: data/grade-ruflo-big.json.)
  • DECISION TAKEN: REAL-USE is the gate of record (consumer-truth); STRICT is a diagnostic.

Retrieval engineering trajectory (ruflo big, 12 Q × 3 vendors) — REAL improvement, principled levers

Stage avg STRICT avg REAL-USE notes
baseline (1 vendor) 49.5 81.8 not comparable (1 lenient vendor)
big, 3 vendors 31.6 68.0 honest 3-vendor floor
+ intent/source tuning 35.0 67.2 whack-a-mole; heuristic ceiling
+ symbol index (ADR-0003) 34.7 67.2 routes to source, but picks referencing file not implementing
+ cross-encoder rerank 49.9 77.1 content-aware; Q2/Q9/Q12 → 95-99
+ selective rerank (skip design/ADR) 55.9 76.7 Q6 recovered to 87; 8/12 questions now 74-99 REAL-USE
  • STRONG now (REAL-USE): Q11 99 · Q12 99 · Q2 98 · Q9 95 · Q1 89 · Q6 87 · Q10 82 · Q3 83 · Q4 74.
  • Stragglers — and WHY (validated, not guessed): Q5 62 (guidance logic is in a compiled WASM kernel), Q7 22 + Q8 30 are SYNTHESIS questions ("what areas do 134 ADRs cover", "memory end-to-end") with NO single answer passage → require the L2 concept layer. Retrieval cannot fix these by construction.
  • PROVEN: the system works end-to-end; implementation questions are handled (74-99); the architecture's L2 layer is empirically necessary (not optional) for overview/playbook questions. 0 hallucinated citations across every run. Levers built: build-symbols.mjs, forge-rerank.mjs (cross-encoder), tuned forge-ask.mjs.

Status update

Layer/lever State
L3 deep source + retrieval ✅ proven (symbol routing + cross-encoder rerank)
Grading (ground-truth + 3-vendor) ✅ proven, ruthless, 0 hallucinated cites
L2 concept/synthesis layer ✅ PROVEN + HARDENED — citation gate enforces ≥2 real refs (rejects ungrounded). 2 accepted @98.
Explainer / presentation ✅ built + visually verified (explainer/index.html, blueprint-terminal design, real numbers)
ruflo bundle ✅ packaged dist/ruvnet-brain.zip (self-contained per ADR-0001)
Repo deep-walks (parallel) ✅ 4/5 T0+T1: ruflo 9,648 · RuVector 28,297 · AgentDB 3,624 · RuLake 1,149 = 42,718 chunks. RuView pending (reuse Cognitum's).
Explainer images ✅ 5 generated (gpt-image-1) + integrated w/ plain-language captions; visually verified. UX-grade in progress.
Ruflo swarm swarm-1782612880654-iup3az (ux-reviewer/synthesizer/grader/builder)
Scale T0/T1 (RuVector, RuView, AgentDB, RuLake) ⬜ after L2 proves ruflo to gate
Explainer + presentation ⬜ after solution complete

L2 synthesis layer — PROVEN with a critical honesty catch (build-l2.mjs)

  • guidance-mechanism: 3 real citations · 3-vendor score 98 · adr-coverage: 1 cite · 98 · memory-end-to-end: 0 cites · 98.
  • The catch (this is the whole ballgame): the 0-citation article scored 98 from the vendor panel. LLM graders pass ungrounded-but-fluent text. ONLY the mechanical ground-truth citation count (0/0) catches it → gate of record MUST be ground-truth citations, not LLM scores (ADR-0002 validated live). L2 generation must ENFORCE ≥2 implemented-code citations or REJECT (memory-end-to-end would be rejected).

COMPLETE ARCHITECTURE PROVEN ON RUFLO (the milestone)

Every question type answerable at gate via the right layer: implementation → L3+symbol+rerank (74-99 REAL-USE); overview/playbook → L2 synthesis (98, when grounded). Ruthless 3-vendor ground-truth grading; 0 hallucinated citations in retrieval; L2 ungrounded-article risk caught by the citation gate. The hard 15-month core is solved in architecture + proof. Remaining = engineering scale: harden L2 (enforce citations) + integrate L2 into corpus (re-embed) + scale to 169 repos + explainer. This is multi-day build effort, not unsolved research.

Honest open risks (carried from red-team)

  • L2 concept synthesis is the highest novel-error surface (unbuilt).
  • Ground-truth grading tooling is unbuilt — the 98 gate is not yet real.
  • RuVector (1.58M LOC) is a much larger build than ruflo; not yet attempted.
  • Single-container still a deferred spike; shipping a zip bundle.

2026-07-09 — Prove-it pass: eight improvements delivered (all evidence pasted below)

Theme: every claim this project makes must be checkable, and every number must be earned. Three separate "reports success, measured nothing" bugs were found and killed.

# Shipped Proof
6 MetaHarness memRetrieve fix + revert-guard oia_audit persisted audit-2026-07-09T04-09-36-769Z → audit_list returned 3 real records (was []) → drift_from_history diffed the baseline. scripts/fix-metaharness-memretrieve.mjs --check fails loudly if npm update -g wipes it. 11 tests.
8 --doctor PROVES grounding Was: `/rvf
7 Footer carries a receipt 🧠 RuvNet Brain jumped in · cited <repo>/<path> — or, truthfully, guidance only, no source read. Never invent a path.
— ADR block no longer fires every prompt 1086 → 0 bytes injected on "what is the capital of France?"
— sync-version --check un-broken Had been red at HEAD on its own moved test fixtures. Still catches a real stray (verified by injecting one).
9 Private-data fence extracted + tested scripts/private-fence.mjs (100% covered). 19 tests replace 11 it.todo. Mutation-verified: reintroduce the QE-0011 fail-open bug → 2 tests go red.
10 Eval flywheel evals/held-out.json — 12 questions, expectations chosen from first principles BEFORE running. npm run eval → grounded 12/12, routed 10/12. Fail-closed proven: no baseline → exit 1; below baseline → exit 1. No LLM judge (a panel once scored a 0-citation answer 98/100).
5 npm publish ruvnet-brain@1.14.0-dev (was 1.6.2-dev). Verified by pulling the tarball back from the public registry.
4 Explainer measurement Deployed. Speed Insights live (200); pageview counter in place, 404s until one dashboard toggle. Old link byte-identical (the 70 people are fine).

Two bugs caught by npm publish --dry-run — before publishing, not after

  1. Offline safety net asked for a Release that never existed. RELEASE_VERSION was derived from the installer's own version and used as the bundle tag: v1.14.0-dev → HTTP 404; the real bundle is v0.5.0-dev → 200. Broken in exactly the situation it exists for.
  2. --doctor would have looped every new user. It loads the verifier from the bundle, and no published bundle contains it. "Not verifiable — re-run the installer" → re-run fetches the same bundle. Fixed by embedding the verifier in the single-file installer (base64, drift-guarded by scripts/embed-verifier.mjs
    • 5 tests). Proven: deleted the verifier from the KB, ran --doctor, it wrote it back byte-identical and printed Grounding PROVEN.

11 — Dogfooding metaharness_evolve: the fitness function was a stub

The .metaharness/ archive from 2026-07-07 declared winner: baseline across 16 variants. That conclusion measured nothing about this repo: the run records show taskId: "mock-1".."mock-5", startedAt: 1970-01-01, stdout: "plan: step 0 … verify: PASS". Synthetic stubs. npm test never ran. The tell was structural — taskSuccess was identical to testPassRate in all 16 rows and 13 landed on exactly 0.6. The promotion gate itself is sound (finalScore > parent + 0.05 AND no testPassRate regression; it correctly refused a regressing variant). The gate was honest; the sensor was a stub.

Two further defects found in the plugin, both "success that isn't":

  • metaharness_evolve MCP tool returned {success: true, degraded: false, exitCode: 0, data: null} while the underlying wrapper returned {degraded: true, reason: "metaharness-darwin-timeout"}. The MCP layer swallows the degraded flag. Trusting it would mean reporting "evolve ran clean" when nothing ran.
  • The wrapper shells out via npx with a 60s default timeout, but --sandbox real runs npm test (~2 min/variant here), so a real-sandbox evolve times out unless the caller passes --timeout-ms. Fixed locally by adding @metaharness/darwin@~0.8.0 as a devDependency (pinned to the plugin's own DARWIN_PIN).

With the binary installed and --timeout-ms 900000, evolve finally ran for real (738s, real npm test output in the traces). It exposed a third defect, the worst of the three:

task: run repository test suite
  exitCode: 1 | durationMs: 120102 | timedOut: true
score: { testPassRate: 0, finalScore: 0.285, promoted: true,
         reason: "promoted: finalScore 0.2850 > parent 0.0000 ... no test regression" }
wrapper: { "improved": true }

The sandbox enforces a 120s per-task cap. Our npm test takes ~116-120s locally (the capability battery runs real KB queries), so it was killed, not failed. testPassRate: 0 is the absence of a measurement, but the scorer reads it as one — and a killed suite was then promoted as champion because the parent scored 0.000. improved: true.

The common structure of all three bugs: the failure path produced a NUMBER instead of an error. A system that cannot distinguish "bad" from "unmeasured" will eventually report the second as the first. (This is exactly why --doctor now separates NOT grounded from not verifiable.)

To make evolve's fitness signal usable on this repo, npm test must finish inside 120s. Options, not yet taken because they change what npm test means and that is Stuart's call: (a) parallelize the capability battery (8 sequential ~10s KB queries dominate the runtime) — no semantic change; (b) move the battery to npm run test:capability and leave npm test as the fast hermetic gate CI already relies on. Until then, treat any metaharness_evolve result on this repo as unmeasured, not as evidence.

Near-miss, recorded honestly

Running scripts/build-concepts.mjs to "verify identical output" overwrote the live KB — kb/ is a symlink to ~/.cache/ruvnet-brain/kb. It regenerated 132 passages over a shipped store of 127; since .rvf vectors are id-mapped to that file, every vector would have mapped to the wrong text. Caught in the diff, restored byte-exact from a pre-run copy, rebuilt the (gitignored) concepts.meta.json from the restored passages, verified id alignment against the idmap, and confirmed retrieval is unchanged (ce=0.201, vec=0.8686, same path/text). A build script is not a read-only verification tool. Separately: the shipped concepts store is stale relative to current primers (127 vs 132) — rebuilding it properly needs a forge-big re-embed, a deliberate release step.

2026-07-09 close — Session re-score (standing order: re-score every session, target ≥95)

Overall 55 → 61. Receipts in AgentDB scorecard-2026-07-09-close. The gate is ARMED: n=120, grounded 100/100 [96.3,100], routed 63/80 [68.6,86.3], abstain 18/20, banner 20/20 — 19 failures recorded verbatim (evals/runs/2026-07-09-baseline.md), scenario stratum weakest. Effectiveness 58→74, reliability 61→72, acting-like-rUv 38→44 (flywheel armed w/ 101 seeded patterns; ADR corpus machine-readable; loop still unbuilt). Token efficiency UNCHANGED 41. CRITICAL: AgentDB corruption RECURRED same-day (idx_bridge_key) — recurring under normal use; re-repaired zero-loss (547 entries). Integrity check is now an every-session ritual item; root-cause (suspected concurrent writers) must land before any autonomy soak claim. Next session: Phase 1 autonomy (run through SPARC), then reranker (P3) before token tax (P2).

2026-07-10 — Re-score: 55 → 74 overall (receipts in AgentDB scorecard-2026-07-10)

Movement: effectiveness 58→82 (n=120 Wilson-gated eval, gate re-PASSED post token-diet), tokens 41→72 (684B/turn, eval-proven free), reliability 61→80 (claims ledger + integration in CI), end-user 54→74 (npm 1.16.0-dev live + autonomy contract), dev-smarter 62→78, corpus 71→76 (gists+nightly proven), acting-like-rUv 38→55 (capped ≤70: AgentDB corrupted 3rd time — REINDEXed ok, 583 rows; flywheel pattern store reads 0; memory search still 0-hit). npm token now SELF-RENEWING (launchd-proven). Honest gap to 95: reranker land+gate, corpus tier-1, token meter, memory-search + corruption root cause, enforced method, Windows, real-user soak.

2026-07-10 — 2.0.0 SHIPPED (all three channels)

git main pushed · npm latest=2.0.0 · Release v2.0.0 live (707MB signed bundle, first releases/latest advance since Jul 1) · issue #4 closed with full receipt note · evening scorecard 55→83 live on both pages · share card + favicon carry 2.0 · security catch: self-update canonical pointed at ruvnet/ (unowned org) — fixed to stuinfla/ everywhere. Tonight: first fully-automated nightly (3:15) + verifier (7:17).