Phase 22: capability truth table + MCP scope freeze + release gates

.github/workflows/release.yml

@@ -63,6 +63,9 @@ jobs:
           npm run eval:web-ai:fixtures
           npm run benchmark:trajectory -- --help
 
+      - name: Release gates (gate:all)
+        run: npm run gate:all
+
       - name: Git diff whitespace check
         run: git diff --check
 

docs/EXTERNAL_CDP.md

@@ -0,0 +1,77 @@
+---
+created: 2026-05-05
+phase: 22
+status: deferred
+tags: [agbrowse, external-cdp, deferred, experimental]
+---
+
+# External / Remote CDP Adapter — Deferred (Experimental)
+
+## Status
+
+**Deferred. Not for production use.**
+
+There is no production-ready external/remote CDP provider in `agbrowse`
+today. Any code, branch, or sketch labeled `external-cdp`, `remote-cdp`, or
+"hosted browser" is **experimental** until this document is replaced with a
+release note that lists:
+
+- a stable connection contract (URL, auth, version negotiation),
+- a session-lifecycle owner (who creates / tears down remote tabs),
+- a policy boundary (allow/deny origins, downloads, clipboard, file access),
+- a trace-evidence guarantee equal to the local-CDP path,
+- and a test suite covering disconnect, slow network, and origin spoofing.
+
+Until that lands, do **not** describe `agbrowse` as supporting hosted, cloud,
+or SaaS browser operation in:
+
+- `README.md`
+- `docs/production-readiness.md`
+- release notes / changelog
+- `structure/CAPABILITY_TRUTH_TABLE.md` "ready" rows
+- public marketing or comparison tables
+
+## Why deferred
+
+The local-CDP path (`skills/browser/`, `web-ai/`) is the supported runtime.
+It has:
+
+- deterministic `browser doctor` cleanup,
+- accessibility snapshot + ref registry,
+- web-AI provider safety policy,
+- trace evidence and source-audit contracts.
+
+A remote/external CDP adapter changes the trust boundary (the browser
+process is not on the user's machine), the failure mode (network partition
+becomes a normal operating condition), and the policy surface (cross-origin
+data egress now happens on a remote box). None of those have a tested,
+versioned contract yet.
+
+Calling such a path "ready" without that contract would mislead users about
+what `agbrowse` guarantees in production.
+
+## How experimental code must be marked
+
+Any commit that introduces or modifies external/remote-CDP code paths must:
+
+1. Carry an `// EXPERIMENTAL: not for production use` marker at the top of
+   each affected source file.
+2. Be guarded by an opt-in flag (env var or `--experimental-external-cdp`)
+   that defaults to off.
+3. Update `structure/CAPABILITY_TRUTH_TABLE.md` to keep External CDP in the
+   `deferred (experimental)` row.
+4. Avoid any user-facing claim of readiness in `README.md` /
+   `docs/production-readiness.md` / release notes.
+
+## Lifting the deferral
+
+Replace this document with `docs/external-cdp.md` describing the production
+contract once all of the following are true:
+
+- Connection / auth / version contract is documented and tested.
+- Policy and trace-evidence parity with local CDP is proven by tests.
+- Truth table row moves to `ready (experimental)` then `ready` only after
+  field validation.
+- A release note explicitly announces the change.
+
+Until then, this file governs.

package.json

@@ -51,6 +51,12 @@
     "test:mcp": "vitest run test/integration/web-ai-mcp-server.test.mjs test/integration/web-ai-policy-mcp.test.mjs test/unit/browser-tool-schema.test.mjs test/unit/web-ai-tool-schema.test.mjs",
     "test:source-audit": "vitest run test/unit/web-ai-source-audit*.test.mjs test/unit/web-ai-answer-artifact.test.mjs",
     "test:release-gates": "bash structure/check-doc-drift.sh && bash structure/verify-counts.sh",
+    "gate:typecheck": "node scripts/release-gates.mjs typecheck",
+    "gate:tests": "node scripts/release-gates.mjs tests",
+    "gate:truth-table-fresh": "node scripts/release-gates.mjs truth-table-fresh",
+    "gate:mcp-scope-frozen": "node scripts/release-gates.mjs mcp-scope-frozen",
+    "gate:no-experimental-in-readme-ready-section": "node scripts/release-gates.mjs no-experimental-in-readme-ready-section",
+    "gate:all": "node scripts/release-gates.mjs",
     "benchmark:trajectory": "node benchmarks/agbrowse/run-task.mjs",
     "release": "bash scripts/release.sh",
     "release:preview": "bash scripts/release-preview.sh",

scripts/release-gates.mjs

@@ -0,0 +1,177 @@
+#!/usr/bin/env node
+/**
+ * Phase 22 named release gates for agbrowse.
+ *
+ * Each gate has a NAME, a CHECK function, and prints PASS / FAIL.
+ * Usage:
+ *   node scripts/release-gates.mjs              # run all gates
+ *   node scripts/release-gates.mjs <gate-name>  # run one gate
+ *
+ * Wired through package.json scripts as `gate:<name>`.
+ */
+import { spawnSync } from 'node:child_process';
+import fs from 'node:fs';
+import path from 'node:path';
+import { fileURLToPath } from 'node:url';
+
+const repoRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
+
+function run(cmd, args, opts = {}) {
+    return spawnSync(cmd, args, {
+        cwd: repoRoot,
+        stdio: opts.stdio || 'pipe',
+        encoding: 'utf8',
+        ...opts,
+    });
+}
+
+function readFile(rel) {
+    return fs.readFileSync(path.join(repoRoot, rel), 'utf8');
+}
+
+const GATES = {
+    'typecheck': {
+        description: 'syntactic + structural integrity (node --check + doc drift)',
+        check() {
+            // agbrowse is .mjs (no TypeScript). Treat node --check on the
+            // public surface plus doc-drift as the equivalent of a typecheck.
+            const targets = [
+                'bin/agbrowse.mjs',
+                'bin/agbrowse-vision-click.mjs',
+                'web-ai/cli.mjs',
+                'web-ai/mcp-server.mjs',
+                'web-ai/browser-tool-schema.mjs',
+                'web-ai/tool-schema.mjs',
+                'scripts/release-gates.mjs',
+            ];
+            for (const rel of targets) {
+                const abs = path.join(repoRoot, rel);
+                if (!fs.existsSync(abs)) continue;
+                const r = run('node', ['--check', abs]);
+                if (r.status !== 0) {
+                    return { ok: false, detail: `node --check failed for ${rel}:\n${(r.stderr || r.stdout || '').slice(-1000)}` };
+                }
+            }
+            const drift = run('bash', ['structure/check-doc-drift.sh']);
+            if (drift.status !== 0) {
+                return { ok: false, detail: `doc drift failed:\n${(drift.stdout || drift.stderr || '').slice(-2000)}` };
+            }
+            return { ok: true, detail: `node --check clean for ${targets.length} entries; doc drift clean` };
+        },
+    },
+    'tests': {
+        description: 'unit + MCP + source-audit + trace-policy tests pass',
+        check() {
+            const suites = ['test:unit', 'test:mcp', 'test:source-audit', 'test:trace-policy'];
+            for (const suite of suites) {
+                const r = run('npm', ['run', suite, '--silent']);
+                if (r.status !== 0) {
+                    return { ok: false, detail: `${suite} failed:\n${(r.stdout || r.stderr || '').slice(-2000)}` };
+                }
+            }
+            return { ok: true, detail: `passed: ${suites.join(', ')}` };
+        },
+    },
+    'truth-table-fresh': {
+        description: 'CAPABILITY_TRUTH_TABLE.md edited within 7 days OR matches code refs',
+        check() {
+            const rel = 'structure/CAPABILITY_TRUTH_TABLE.md';
+            const abs = path.join(repoRoot, rel);
+            if (!fs.existsSync(abs)) return { ok: false, detail: `${rel} missing` };
+            const stat = fs.statSync(abs);
+            const ageMs = Date.now() - stat.mtimeMs;
+            const ageDays = ageMs / (1000 * 60 * 60 * 24);
+            if (ageDays <= 7) {
+                return { ok: true, detail: `truth table ${ageDays.toFixed(2)}d old` };
+            }
+            // fallback: ensure every frozen MCP tool name appears in the table
+            const text = readFile(rel);
+            const required = ['browser_snapshot', 'browser_click_ref', 'answerArtifact', 'sourceAudit'];
+            for (const term of required) {
+                if (!text.includes(term)) {
+                    return { ok: false, detail: `truth table stale (${ageDays.toFixed(1)}d) and missing ${term}` };
+                }
+            }
+            return { ok: true, detail: `truth table ${ageDays.toFixed(1)}d old but matches required terms` };
+        },
+    },
+    'mcp-scope-frozen': {
+        description: 'only the 2 frozen browser MCP tools are registered',
+        check() {
+            const text = readFile('web-ai/browser-tool-schema.mjs');
+            const matches = [...text.matchAll(/^\s{4}(browser_[a-z_]+):\s*{/gm)].map((m) => m[1]);
+            const expected = ['browser_snapshot', 'browser_click_ref'];
+            if (matches.length !== 2 || matches[0] !== expected[0] || matches[1] !== expected[1]) {
+                return { ok: false, detail: `expected ${expected.join(',')}, found ${matches.join(',') || '(none)'}` };
+            }
+            return { ok: true, detail: 'browser MCP scope frozen at browser_snapshot, browser_click_ref' };
+        },
+    },
+    'no-experimental-in-readme-ready-section': {
+        description: 'README "ready" claims do not include external CDP or unimplemented MCP tools',
+        check() {
+            const readme = readFile('README.md');
+            // capture content from a "ready" / "Production" / "Supported" header up to next ##
+            const sections = readme.split(/\n##\s+/);
+            const offending = [];
+            const forbiddenInReady = [
+                /external[-\s]?cdp/i,
+                /remote[-\s]?cdp/i,
+                /hosted browser/i,
+                /browser_type_ref/,
+                /browser_navigate/,
+                /browser_screenshot/,
+                /browser_back/,
+                /browser_forward/,
+                /browser_reload/,
+                /browser_wait_for/,
+                /browser_extract_text/,
+            ];
+            for (const sec of sections) {
+                const head = sec.split('\n', 1)[0].toLowerCase();
+                const isReady = head.includes('ready') || head.includes('production') || head.includes('supported');
+                const isExperimentalSection = head.includes('experimental') || head.includes('deferred') || head.includes('out of scope');
+                if (isReady && !isExperimentalSection) {
+                    for (const pat of forbiddenInReady) {
+                        if (pat.test(sec)) offending.push(`${head} :: ${pat}`);
+                    }
+                }
+            }
+            if (offending.length > 0) {
+                return { ok: false, detail: `forbidden terms in ready section:\n${offending.join('\n')}` };
+            }
+            return { ok: true, detail: 'README ready sections do not advertise experimental/unimplemented surfaces' };
+        },
+    },
+};
+
+function printResult(name, result) {
+    const status = result.ok ? 'PASS' : 'FAIL';
+    process.stdout.write(`[${status}] gate:${name} — ${GATES[name].description}\n`);
+    if (result.detail) process.stdout.write(`        ${result.detail.replace(/\n/g, '\n        ')}\n`);
+}
+
+function main() {
+    const target = process.argv[2];
+    const names = target ? [target] : Object.keys(GATES);
+    let failed = 0;
+    for (const name of names) {
+        if (!GATES[name]) {
+            process.stdout.write(`[FAIL] gate:${name} — unknown gate\n`);
+            failed += 1;
+            continue;
+        }
+        let result;
+        try {
+            result = GATES[name].check();
+        } catch (err) {
+            result = { ok: false, detail: `threw: ${err.message}` };
+        }
+        printResult(name, result);
+        if (!result.ok) failed += 1;
+    }
+    process.stdout.write(failed === 0 ? `\nAll ${names.length} gate(s) passed.\n` : `\n${failed}/${names.length} gate(s) FAILED.\n`);
+    process.exit(failed === 0 ? 0 : 1);
+}
+
+main();

scripts/release-preview.sh

@@ -60,6 +60,9 @@ npm run test:eval-fixtures
 npm run eval:web-ai:fixtures
 npm run benchmark:trajectory -- --help >/dev/null
 
+echo "Running release gates (gate:all)..."
+npm run gate:all
+
 echo "Checking diff whitespace..."
 git diff --check
 

scripts/release.sh

@@ -65,6 +65,9 @@ npm run test:eval-fixtures
 npm run eval:web-ai:fixtures
 npm run benchmark:trajectory -- --help >/dev/null
 
+echo "Running release gates (gate:all)..."
+npm run gate:all
+
 echo "Checking diff whitespace..."
 git diff --check
 

structure/CAPABILITY_TRUTH_TABLE.md

@@ -0,0 +1,66 @@
+---
+created: 2026-05-05
+phase: 22
+tags: [agbrowse, truth-table, release-claims, source-of-truth]
+aliases: [agbrowse capability truth table]
+---
+
+# agbrowse Capability Truth Table
+
+This is the **single source of truth** for capability status across `agbrowse`
+and its `cli-jaw` mirror. Phase 22 introduces this table to keep public claims
+aligned with code, tests, and the cli-jaw mirror surface. Update this file in
+the same commit as any capability or claim change.
+
+Status legend:
+
+- `ready` — implementation, tests, and public docs all agree.
+- `beta` — implementation exists; depends on live provider UI / accounts.
+- `experimental` — opt-in, narrow scope, no production claim.
+- `deferred` — explicitly not implemented; do not market.
+
+`Mirror In cli-jaw` describes the equivalent surface in `cli-jaw` if any. A
+mirror entry of `n/a` means the capability is intentionally agbrowse-only.
+
+| Capability | Status | Code Location | Tests | Mirror In cli-jaw |
+| --- | --- | --- | --- | --- |
+| Browser runtime cleanup / `doctor` | ready | `web-ai/doctor.mjs`, `skills/browser/browser.mjs` | `test/integration/web-ai-doctor*.test.mjs`, `test/unit/web-ai-doctor.test.mjs` | `src/browser/runtime/*` (cleanup); doctor surface re-exported via cli-jaw browser command. ready in cli-jaw. |
+| ChatGPT web-AI resolver | beta | `web-ai/chatgpt.mjs`, `web-ai/chatgpt-composer.mjs`, `web-ai/chatgpt-model.mjs` | `test/unit/web-ai-chatgpt*.test.mjs`, fixture evals under `test/fixtures/provider-dom/` | `src/browser/web-ai/chatgpt.ts` — beta in cli-jaw. |
+| Gemini web-AI resolver | beta | `web-ai/gemini-live.mjs`, `web-ai/gemini-model.mjs` | `test/unit/web-ai-gemini*.test.mjs` | not mirrored; cli-jaw delegates via agbrowse. n/a in cli-jaw. |
+| Grok web-AI resolver | beta | `web-ai/grok-live.mjs`, `web-ai/grok-model.mjs` | `test/unit/web-ai-grok*.test.mjs` | not mirrored. n/a in cli-jaw. |
+| Action-intent / semantic target resolver (incl. `send.click`) | ready | `web-ai/action-intent.mjs`, `web-ai/target-resolver.mjs`, `web-ai/self-heal.mjs` | `test/unit/web-ai-action-intent.test.mjs`, `test/unit/web-ai-target-resolver.test.mjs` | `src/browser/web-ai/action-intent.ts`, `src/browser/web-ai/target-resolver.ts`. ready in cli-jaw. |
+| `answerArtifact` on completed answers | ready | `web-ai/answer-artifact.mjs` | `test/unit/web-ai-answer-artifact.test.mjs` | `src/browser/web-ai/answer-artifact.ts`, `tests/unit/browser-web-ai-answer-artifact.test.ts`. ready in cli-jaw. |
+| `sourceAudit` (`--require-source-audit`, ratio/scope/date flags) | ready | `web-ai/source-audit.mjs`, CLI surface in `web-ai/cli.mjs` | `test/unit/web-ai-source-audit*.test.mjs` | `src/browser/web-ai/source-audit.ts`, CLI flags via `src/browser/web-ai/index.ts`, HTTP via `src/routes/browser.ts`, `tests/unit/browser-web-ai-source-audit.test.ts`. ready in cli-jaw. |
+| MCP tool: `browser_snapshot` | ready (frozen scope) | `web-ai/browser-tool-schema.mjs`, `web-ai/mcp-server.mjs` | `test/unit/browser-tool-schema.test.mjs`, `test/integration/web-ai-mcp-server.test.mjs` | n/a in cli-jaw (cli-jaw does not expose browser MCP tools). |
+| MCP tool: `browser_click_ref` | ready (frozen scope) | `web-ai/browser-tool-schema.mjs`, `web-ai/mcp-server.mjs` | `test/unit/browser-tool-schema.test.mjs`, `test/integration/web-ai-mcp-server.test.mjs` | n/a in cli-jaw. |
+| MCP tools: `browser_type_ref`, `browser_navigate`, `browser_back`, `browser_forward`, `browser_reload`, `browser_wait_for`, `browser_screenshot`, `browser_extract_text` | deferred (`not-implemented`) | listed in `web-ai/browser-tool-schema.mjs` `NOT_IMPLEMENTED_BROWSER_TOOLS` | regression test in `test/unit/browser-tool-schema.test.mjs` | n/a. |
+| Web-AI MCP tools (`web_ai_*`) | beta | `web-ai/tool-schema.mjs`, `web-ai/mcp-server.mjs` | `test/integration/web-ai-mcp-server.test.mjs`, `test/unit/web-ai-tool-schema.test.mjs` | n/a. |
+| Policy enforcement (`policy/*`) | ready | `web-ai/policy/` | `test/unit/web-ai-policy*.test.mjs`, `test/integration/web-ai-policy-*.test.mjs` | partial mirror via cli-jaw browser route policy; agbrowse remains source. |
+| Trace evidence (Phase 12) | ready | `web-ai/trace/`, `web-ai/trace-persistence.mjs`, `scripts/render-trace-report.mjs` | `test/unit/web-ai-trace*.test.mjs` | n/a; cli-jaw does not mirror trace. |
+| External / remote CDP adapter | deferred (experimental) | _no production code_; `docs/EXTERNAL_CDP.md` documents the deferral | none | deferred. See `docs/EXTERNAL_CDP.md` in both repos. |
+| Benchmark trajectory writer | ready (offline bundle only) | `benchmarks/agbrowse/trajectory.mjs`, `benchmarks/agbrowse/run-task.mjs` | `test/unit/benchmark-trajectory.test.mjs` if present; smoke via `npm run benchmark:trajectory -- --help` | planned — cli-jaw consumes agbrowse trajectory bundles; no native writer. |
+| Benchmark leaderboard / score claim | deferred | n/a | n/a | deferred. |
+| Release gates (named) | ready | `scripts/release.sh`, `scripts/release-preview.sh`, `scripts/release-gates.mjs` (Phase 22) | `npm run gate:*` series | mirrored via cli-jaw `scripts/release-gates.mjs`. ready in cli-jaw. |
+
+## Mirror Rules
+
+- A `ready` claim in agbrowse does **not** automatically mean `ready` in
+  `cli-jaw`. The cli-jaw column above governs cross-repo claims.
+- New capability or claim ⇒ update this table and the equivalent in cli-jaw in
+  the same commit (`gate:truth-table-fresh` enforces ≤7 day staleness).
+- Frozen MCP scope: only the two `browser_*` tools above may be registered
+  without an explicit table change (`gate:mcp-scope-frozen`).
+
+## Forbidden Claims
+
+- No `ready` claim for hosted/cloud, remote/external CDP, or stealth flows.
+- No leaderboard or competitor benchmark score until a fixed
+  model/planner/environment/task set lands.
+- No `ready` MCP claim beyond the two frozen tools.
+
+## Cross-References
+
+- Phase truth table: [phase_status.md](phase_status.md)
+- Release gate checklist: [release_gates.md](release_gates.md)
+- External CDP deferral: [../docs/EXTERNAL_CDP.md](../docs/EXTERNAL_CDP.md)
+- Phase 22 plan: `cli-jaw/devlog/_plan/260505_browser_runtime_phase22/22_agbrowse_parity_closeout.md`

structure/INDEX.md

@@ -22,6 +22,7 @@ aliases: [agbrowse 구조 허브, agbrowse source of truth, agbrowse architectur
 | [commands.md](commands.md) | `agbrowse` root CLI와 `web-ai` command surface | `skills/browser/browser.mjs`, `web-ai/cli.mjs` |
 | [runtime_contracts.md](runtime_contracts.md) | sessions, tabs, provider, policy, trace, MCP, eval runtime 계약 | `web-ai/`, `skills/browser/`, `devlog/13_phase12_trace_replay.md` 이후 |
 | [release_gates.md](release_gates.md) | ready/beta/experimental 라벨과 release 전 검증 | `package.json`, `scripts/release.sh`, `.github/workflows/release.yml` |
+| [CAPABILITY_TRUTH_TABLE.md](CAPABILITY_TRUTH_TABLE.md) | Phase 22 capability/cli-jaw mirror truth table (single source of truth) | `web-ai/`, `cli-jaw/structure/CAPABILITY_TRUTH_TABLE.md` |
 | [phase_status.md](phase_status.md) | Phase 11+ 구현/미러/claim 상태 truth table | `devlog/00_index.md`, `web-ai/`, `cli-jaw` mirror |
 | [check-doc-drift.sh](check-doc-drift.sh) | 구조 문서의 최소 drift 검사 | `package.json`, `README.md`, `structure/*.md` |
 | [verify-counts.sh](verify-counts.sh) | `str_func.md`의 파일 수/라인 수 스냅샷 검증 | `structure/str_func.md`, live source tree |