Skip to content

SPEC.md

Latest commit

 

History

History
654 lines (530 loc) · 30.6 KB

SPEC.md

File metadata and controls

654 lines (530 loc) · 30.6 KB
Raw

bitrouter-agent — v1 Build Specification

Status: build spec for a fresh repository. This document is the single source of truth for building v1. It is written to be handed to an agent that loops until every item in §19 Acceptance Criteria is green. Read §0 Build-Loop Protocol first.

0. Build-Loop Protocol

You are building bitrouter-agent v1 from an empty repository. Work milestone by milestone in the order given in §18. After each milestone:

  1. Run the checks: pnpm typecheck && pnpm lint && pnpm test.
  2. Run the end-to-end smoke (§20) once that milestone makes it runnable.
  3. Commit with a conventional-commit message (feat:, chore:, test: …), title ≤ 60 chars. Branch off main; do not push unless asked.
  4. Re-read §19. The loop is done when, and only when, every acceptance box can be checked and the checks in step 1–2 pass on a clean tree. Until then, pick the next unmet item and continue.

Rules that hold for the whole build:

  • Zero-write to user code. The agent this tool runs must never modify or create files inside the user's existing source. It may only create the new artifact files in §3. Enforce this by tool absence (ship no edit/write-to-source tool), not by a scanner. See §9.
  • pi-only. Do not add @anthropic-ai/claude-agent-sdk. The agent loop is the pi SDK. See §5 and §7.
  • Verify external APIs against installed types. Code sketches below match pi ~0.79 (the version @posthog/wizard ships against). Pin the version; if a signature differs, trust the installed .d.ts, not this doc.
  • The sibling checkout at ../bitrouter is the source of truth for the config schema, example config, registry prices, and the /bitrouter skill. Reference it at build time (see §12, §13).

1. Product Summary

bitrouter-agent is a CLI wizard — modeled on npx @posthog/wizard — that a developer runs inside their production agentic codebase. It statically analyzes how the codebase spends LLM tokens and produces three artifacts that help the user adopt BitRouter (the cost-optimizing LLM gateway/router, the sibling project) to cut that spend. It is built on the pi coding-agent SDK (@earendil-works/pi-coding-agent) and routes its own LLM calls through BitRouter via the @bitrouter/pi provider package.

The UX north star is PostHog's wizard: a focused, mostly-scripted terminal flow where an embedded coding agent does the codebase-specific reasoning, gated by clear consent, ending in concrete artifacts.

2. Scope — v1 In / Out

In (v1 = "T0 · Instant", static cold-start):

  • Standalone CLI, distributed as an npm package (npx @bitrouter/agent).
  • Static analysis of the user's repo by a pi agent, read-only.
  • Three output artifacts: an audit, a savings estimate, and an MVP bitrouter.yaml (§3).
  • The agent routed through BitRouter via @bitrouter/pi (§8).
  • A single consent disclosure before analysis (§14).
  • A functional (not necessarily fancy) terminal UI (§16).

Out (design the seams, do not build):

  • Any write to the user's existing files (env swaps, code instrumentation). v1 instructs via comments in the generated yaml only.
  • T2 · Cloud upload / bitrouter-mcp connection / advanced optimization.
  • @anthropic-ai/claude-agent-sdk, the "anthropic" harness, orchestrator mode, magicast/recast.
  • Multi-framework special-casing beyond LLM-SDK detection.

Next milestone, spec'd but after v1 core (T1 · Watch): a watch mode that reads the local running daemon's telemetry (bitrouter.db metering + /metrics) to upgrade the estimate from a rate-card to observed dollars. Spec'd in §4 and §18 M6; not part of the v1 acceptance gate.

3. Deliverables

All artifacts are new files written into the target repo's working dir (or --out-dir). Writing these is the only filesystem mutation allowed.

3.1 bitrouter.yaml — the MVP policy

A valid BitRouter config (§13) that:

  • declares the providers the codebase already uses (keys as ${VAR} placeholders — never real secrets),
  • defines virtual models with strategy: priority fallback chains that route routine call-classes to a cheap open model and keep the current frontier model as the escalation/fallback,
  • carries the OPENAI_BASE_URL=http://localhost:4356 adoption step as comments, not applied edits,
  • passes validation (§13.3).

3.2 bitrouter-audit.md — the audit + estimate (human-readable)

A Markdown report containing:

  • Detected setup: SDK/harness, provider(s), and every model id found, each with its call-sites (path:line) and an inferred call-class (routine | complex).
  • Loop shape: sub-agents, tool calls, MCP servers, retry/fallback wiring observed.
  • Savings estimate (rate card): for each frontier model in use, the current $/Mtok vs the cheapest open-weight equivalent, the routine-share estimate, and a blended % reduction. Framed as a rate comparison / scenario — never a fabricated absolute dollar bill (§12).
  • Recommended routing: the rationale behind the bitrouter.yaml choices.

3.3 .bitrouter/analysis.json — machine-readable analysis (source of truth)

Structured JSON that both bitrouter-audit.md renders from and the future watch mode diffs observed telemetry against. Minimum shape:

{
  "schemaVersion": 1,
  "detectedAt": "<ISO8601>",           // injected by host, not the agent
  "harness": "openai-sdk | anthropic-sdk | claude-code | langchain | custom",
  "models": [
    { "id": "anthropic/claude-opus-4.6", "callSites": ["src/agent.ts:42"],
      "callClass": "complex", "estCallShare": 0.2 }
  ],
  "estimate": {
    "perModel": [
      { "current": "anthropic/claude-opus-4.6", "currentRate": {"in":5,"out":25},
        "openAlt": "alibaba/qwen3-coder", "altRate": {"in":0.4,"out":1.6},
        "routineShare": 0.6 }
    ],
    "blendedReductionPct": 0.0        // computed, see §12
  }
}

4. Feedback Tiers

The tier model resolves the cold-start problem (no traffic ⇒ no real numbers). v1 builds T0 only; the architecture must leave clean seams for T1/T2.

Tier v1? Trigger Data source Output upgrade
T0 · Instant ✅ build at install, 0 traffic static analysis + registry prices rate-card estimate
T1 · Watch ⬜ next (M6) first N metered runs / few days local bitrouter.db metering + /metrics observed absolute $ + per-hop attribution + tightened policy diff
T2 · Cloud ❌ v2 explicit consent upload via bitrouter-mcp cross-run advisor self-tuning

Seam requirements (must exist in v1):

  • The analysis engine emits .bitrouter/analysis.json so T1 has something to diff against.
  • The tool registry is capability-partitioned (readonly set vs a write set that is empty in v1) so T1/T2 add a tool behind a flag, not a refactor. The first future write tool is a telemetry-wiring one (set OTEL_EXPORTER_OTLP_ENDPOINT / add an observe: block), never a code edit.

5. Architecture

Layered, pi-only, per-concern (mirrors the reusable half of @posthog/wizard, minus everything the zero-write decision removes).

bin.ts                      # node ≥22 preflight + Wizard.use(...).init()
src/wizard.ts               # yargs wrapper + global options   (lift from wizard)
src/commands/               # command.ts (Command iface) + one file per command
  command.ts                #   the Command → yargs adapter    (lift from wizard)
  init.ts                   #   $0 default: run all three deliverables
  audit.ts  estimate.ts     #   focused single-artifact entrypoints
src/programs/               # one "program" per deliverable: prompt + toolset + runner
  run-analysis.ts           #   the scripted spine (audit → estimate → yaml)
src/agent/                  # pi embedding
  session.ts                #   createAgentSession wiring (§7)
  provider.ts               #   @bitrouter/pi bootstrap + model pick (§8)
  tools/                    #   read-only + emit_* custom tools (§9)
  prompts.ts                #   system + phase prompts (§10)
src/detection/              # host-side pre-agent scan (§11)
src/pricing/                # registry snapshot + rate-card math (§12)
src/yaml/                   # bitrouter.yaml builder + validator (§13)
src/ui/                     # WizardUI interface + minimal impl (§16)
src/config.ts               # resolved run config (cwd, target, model, flags)
scripts/sync-registry.ts    # build-time: snapshot ../bitrouter/registry → bundled JSON
fixtures/sample-agent-app/  # a tiny fake agentic repo for e2e (§20)

Control flow: bin.ts → Wizard(yargs) → a Command.handler → runAnalysis(config) → mount UI → host-side detection → bootstrap pi session (provider + tools + prompts) → scripted spine: prompt(AUDIT) → prompt(ESTIMATE) → prompt(YAML), each emitting its artifact → summary → exit.

6. Tech Stack & Dependencies

  • Runtime: Node ≥ 22.22 (match pi). Language: TypeScript, ESM ("type": "module"). Package manager: pnpm. Build: tsdown (or tsup). Test: vitest. Lint: eslint + prettier.
  • Package: name @bitrouter/agent; bin: { "bitrouter-agent": "dist/bin.js" }.

Dependencies (use):

Package Why
@earendil-works/pi-coding-agent the agent loop + defineTool + resource loader (pin ~0.79)
@bitrouter/pi registers bitrouter as the agent's model provider; also bundles the /bitrouter skill
typebox tool parameter schemas (Type.Object) — pi's schema lib
yargs CLI parsing
fast-glob host-side file scanning
yaml (or js-yaml) build + serialize bitrouter.yaml
ink + react terminal UI (may defer rich screens — see §16)
semver node-version preflight

Do NOT add: @anthropic-ai/claude-agent-sdk, magicast, recast, langchain (detect its usage, don't depend on it). If you find yourself reaching for the Claude Agent SDK, re-read §17.

7. pi Embedding (the core wiring)

Battle-tested shape from @posthog/wizard's pi harness (src/lib/agent/runner/harness/pi/index.ts), adapted for read-only + our provider. The agent loop is session.prompt() — not runPrintMode/runRpcMode.

import {
  createAgentSession, DefaultResourceLoader, SessionManager,
} from '@earendil-works/pi-coding-agent';

export async function buildSession(cfg: RunConfig, ui: WizardUI) {
  const model = await pickAgentModel(cfg);          // §8 — a "bitrouter/<id>" id

  const resourceLoader = new DefaultResourceLoader({
    cwd: cfg.installDir,
    // Load exactly ONE extension: the bitrouter provider (+ its bundled skill).
    additionalExtensionPaths: [bitrouterProviderExtensionPath()],  // §8
    systemPromptOverride: () => SYSTEM_PROMPT,       // §10
    noContextFiles: true, noThemes: true, noPromptTemplates: true,
  });

  const { session } = await createAgentSession({
    model,
    cwd: cfg.installDir,
    sessionManager: SessionManager.inMemory(cfg.installDir),
    resourceLoader,
    noTools: 'builtin',                 // disable pi built-ins…
    customTools: buildTools(cfg, ui),   // …re-register OUR read-only + emit set (§9)
  });

  await session.bindExtensions({});     // fires session_start → provider registers

  // Progress/telemetry: forward pi events to the UI.
  session.subscribe((e) => {
    if (e.type === 'tool_execution_start') ui.onToolStart(e);
    if (e.type === 'tool_execution_end')   ui.onToolEnd(e);
    if (e.type === 'message_end')          ui.onAssistantText(extractText(e));
  });

  return session;
}

Scripted spine (src/programs/run-analysis.ts). Because pi ends a run on any tool-less turn, use one discrete prompt() per deliverable — deterministic, and it sidesteps the "continue nudge" loop the wizard needs for open-ended runs:

await session.prompt(AUDIT_PROMPT);      // agent explores repo, calls emit_audit
await session.prompt(ESTIMATE_PROMPT);   // agent calls registry_price_lookup + emit_estimate
await session.prompt(YAML_PROMPT);       // agent calls emit_bitrouter_yaml
const stats = await session.getSessionStats();   // token/cost of the wizard's own run

Each phase prompt instructs the agent to finish by calling exactly one emit_* tool; the tool's success is the phase's completion signal.

8. Model Provider Bootstrap (@bitrouter/pi)

The agent's own model is BitRouter-routed. @bitrouter/pi is a pi extension that discovers models (with cost) from GET ${baseUrl}/models and calls pi.registerProvider("bitrouter", …).

  • Configured by env: BITROUTER_TARGET (local default → http://127.0.0.1:4356/v1; cloud → https://api.bitrouter.ai/v1), BITROUTER_BASE_URL (override), BITROUTER_API_KEY (brvk_, local only; cloud reuses the daemon credential file from bitrouter auth login).
  • bitrouterProviderExtensionPath() resolves the extension entry from the installed package (verify the exact subpath against the installed package, e.g. require.resolve('@bitrouter/pi/extensions/bitrouter')).

Bootstrapping order: the provider registers during bindExtensions, but createAgentSession needs a model up front. So pickAgentModel(cfg) resolves the id host-side first: call GET ${baseUrl}/models, pick a capable model (do not reflexively pick the cheapest — policy generation is real reasoning; prefer a strong model, configurable via --model), pass it in, then the extension registers the provider so the id resolves at bind.

Ensure BitRouter is reachable. If /models is unreachable in local mode, the wizard should attempt bitrouter start (zero-config BYOK) or surface a clear "run bitrouter start / bitrouter auth login" error and exit — do not fall back to a direct provider (keeps one path for T0 and T1, and standing up BitRouter is step 1 of adoption anyway).

9. Agent Tools (read-only + emit)

Tools are pi defineTool with typebox schemas. Partition into two sets; v1 registers only the readonly set + the emit set. The write set is empty.

Read-only (exploration): re-register pi built-ins read, grep, find, ls. No bash, no edit, no write. (If a bash-like capability is ever needed, it must be non-mutating and env-scrubbed — out of v1 scope.)

Emit (the only mutations — each writes a new artifact file):

Tool Params (typebox) Behavior
registry_price_lookup { modelId: string } returns { current, cheapestOpenAlt } rates from the pricing engine (§12). Read-only helper.
emit_audit_report { markdown: string, analysis: object } writes bitrouter-audit.md and .bitrouter/analysis.json (host stamps detectedAt).
emit_estimate { perModel: [...], routineShares: {...} } merges estimate into .bitrouter/analysis.json; host computes blendedReductionPct (§12) so the math is trusted, not model-guessed.
emit_bitrouter_yaml { yaml: string } validates (§13.3) then writes bitrouter.yaml; returns validation errors to the agent to retry on failure.

emit_* tools resolve/normalize paths against cfg.outDir, refuse to overwrite outside it, and must reject any attempt to target an existing user file. Keep the money math (blendedReductionPct) and yaml validation in host code, not in the model's free text — the agent supplies inputs; the host computes and validates.

10. Agent Prompts

src/agent/prompts.ts exports a SYSTEM_PROMPT and three phase prompts.

  • SYSTEM_PROMPT — role ("you analyze an agentic codebase to help the user adopt BitRouter"), the read-only contract (you have no ability to modify the user's code; produce artifacts only), the call-class rubric (routine = file reads, summarization, formatting, scaffolding, retries; complex = planning, multi-step reasoning, code generation), and the finish discipline (end each phase by calling its single emit_* tool). The /bitrouter skill (bundled via @bitrouter/pi) supplies BitRouter config facts — do not restate them here.
  • AUDIT_PROMPT — "explore the repo with read/grep/find/ls; identify the SDK/harness, every model id and its call-sites, the loop shape; classify each call-site; call emit_audit_report."
  • ESTIMATE_PROMPT — "for each frontier model found, call registry_price_lookup; estimate the routine share per model; call emit_estimate."
  • YAML_PROMPT — "produce an MVP bitrouter.yaml (providers with ${VAR}, virtual models with strategy: priority chains routing routine→open, frontier as fallback; the OPENAI_BASE_URL swap as comments); call emit_bitrouter_yaml; if it returns validation errors, fix and re-emit."

Prompts must be unit-testable as plain strings (no ink/React imports).

11. Host-side Detection

Fast, deterministic pre-agent scan (mirrors @posthog/wizard src/lib/detection/features.ts) that seeds the agent so it doesn't start blind.

  • Parse package.json / requirements.txt / pyproject.toml / go.mod for known LLM SDKs (openai, @anthropic-ai/sdk, @google/genai, langchain, llamaindex, litellm, …) → harness guess.
  • fast-glob for agent-runtime config (e.g. .claude/, codex config, a bitrouter.yaml already present) with sane ignores (node_modules, dist, .git, build).
  • Output a DetectionResult handed into AUDIT_PROMPT as grounding. Detection never writes anything.

12. Estimate Engine

src/pricing/. Honest cold-start framing: output a rate card, never an invented dollar total.

  • Price source, in order: (1) live GET ${baseUrl}/models from the running daemon (authoritative, current, includes cost); (2) a bundled snapshot pricing/registry-snapshot.json generated at build time by scripts/sync-registry.ts from ../bitrouter/registry/.
  • Model metadata (open_weights, family) comes from ../bitrouter/registry/models/*.yaml (snapshot it too). The provider price shape lives in ../bitrouter/registry/providers/*.yaml — read the real files for the exact nesting; it is roughly: 
    - id: openai/gpt-5.4
  # …
  input_tokens:  { no_cache: 2.5, cache_read: 0.25 }   # $/Mtok
  output_tokens: { text: 15 }                          # $/Mtok
  • "Cheapest open equivalent": among models with open_weights: true in the same capability tier as the frontier model in use, pick the one with the lowest cheapest-provider blended rate.
  • blendedReductionPct (computed in host code): 1 − (routineShare · openBlended + (1−routineShare) · currentBlended) / currentBlended where blended = 0.75·in + 0.25·out (or a documented input/output weight). Also surface the raw before/after $/Mtok per model and the assumed routineShare so the number is transparent, not magic.

13. bitrouter.yaml Output Contract

13.1 Shape

Match ../bitrouter/examples/bitrouter.yaml. Minimum:

# yaml-language-server: $schema=https://bitrouter.dev/schema/v<VERSION>/config.schema.json
#
# Generated by bitrouter-agent. To adopt: point your harness at the router —
#   OPENAI_BASE_URL=http://localhost:4356        # ← the one change you make
# then: bitrouter start
server:
  listen: 127.0.0.1:4356
providers:
  anthropic:
    api_base: https://api.anthropic.com
    api_key: ${ANTHROPIC_API_KEY}          # placeholder — resolved from env, never written
    models: [ { id: claude-opus-4.6 } ]
  alibaba_us:
    # …cheapest open provider for the chosen routine model…
models:
  # A virtual model: routine work → open; escalates to the frontier model.
  smart:
    strategy: priority
    endpoints:
      - { provider: alibaba_us, service_id: qwen3-coder }
      - { provider: anthropic,  service_id: claude-opus-4.6 }

13.2 Rules

  • Every secret is a ${VAR} placeholder. Never emit a real key, even if one is visible in the repo/env.
  • The adoption instructions (OPENAI_BASE_URL swap, bitrouter start) are comments — no applied changes (zero-write).
  • Provider ids, model ids, and price data come from the registry — do not invent provider names.

13.3 Validation

emit_bitrouter_yaml validates before writing:

  1. If the bitrouter binary is on PATH: shell bitrouter config validate -c <tmpfile> (the real parser — preferred).
  2. Else: validate against the bundled JSON schema (../bitrouter/dist/schema/bitrouter.config.schema.json, snapshot it into the package). On failure, return the errors to the agent for a retry; never write an invalid file.

14. Consent Gate

Even local cold-start sends code snippets to an LLM to analyze them. Before the first prompt(), show one disclosure and get an explicit choice (the analog of the wizard's AI opt-in gate):

  • What leaves the machine: code excerpts the agent reads, sent to the model via BitRouter to <target> (local daemon → your configured upstreams / BYOK, or cloud). No artifacts are uploaded in v1.
  • Choice: proceed with local (BYOK/your keys) or cloud; or abort.
  • Persist nothing sensitive; honor a --yes/non-interactive flag for CI that documents the same disclosure in output.

15. CLI Surface

Lift the command framework from @posthog/wizard (src/wizard.ts + src/commands/command.ts): Wizard.use(cmd…).init() over yargs, env-prefixed BITROUTER_AGENT_* options, .strictCommands(), a clean .fail().

Command Effect
bitrouter-agent ($0 = init) run all three deliverables
bitrouter-agent audit audit + estimate only (no yaml)
bitrouter-agent estimate estimate only
(reserved) bitrouter-agent watch T1, milestone M6 — not v1

Global flags: --out-dir, --model, --target {local,cloud}, --yes (non-interactive), --debug, --version, --help.

16. UI

Abstract behind a WizardUI interface (onToolStart/onToolEnd/onAssistantText/ askConsent/summary/spinner) with two implementations, so the runner is UI-agnostic (this is how the wizard supports Ink + headless behind one iface):

  • v1 required: a functional line-based UI — a spinner, per-phase progress, the consent prompt, and a final summary listing the three artifacts + the headline blended-% and the wizard's own run cost (from getSessionStats).
  • v1 required: a --yes non-interactive/logging UI for CI.
  • Later (polish, not in the v1 gate): a rich Ink/React TUI (screens, a live progress list, a token/cost HUD) modeled on the wizard's src/ui/tui/.

Ship ink+react as deps so the polish milestone doesn't need a dependency change, but the v1 acceptance gate only requires the functional UI.

17. Reuse Map (@posthog/wizard)

Study github.com/PostHog/wizard (MIT). It is built on the same pi stack, so it is a direct reference.

Lift (adapt near-verbatim):

  • src/wizard.ts + src/commands/command.ts — command framework.
  • src/lib/agent/runner/harness/pi/index.ts — the createAgentSession + subscribe + prompt wiring.
  • src/lib/agent/runner/harness/pi/tools.ts — defineTool + typebox tool shape.
  • src/lib/agent/token-pricing.ts — the exact-match $/Mtok table pattern (ours sources from the registry, not a hardcoded table).
  • src/lib/detection/features.ts — LLM-SDK detection.

Skip entirely (the zero-write dividend): src/lib/agent/runner/harness/pi/security.ts (YARA/permission fence — unneeded with no write tools), wizardCanUseTool, the secret vault, .claude/settings backup, env-scrubbed bash, magicast/ recast (they are in the wizard's package.json but unused), the entire harness/anthropic/ tree, createSdkMcpServer (Claude Agent SDK), the switchboard harness axis, and the orchestrator sequence.

18. Build Milestones

Each milestone ends with checks + commit (§0).

  • M0 — Scaffold. package.json (deps §6, bin, scripts), tsconfig, tsdown/vitest/eslint config, bin.ts node preflight, src/wizard.ts + src/commands/command.ts, an init command that prints help. DoD: pnpm build && node dist/bin.js --help lists the commands.
  • M1 — Config + detection. src/config.ts, src/detection/. DoD: detection returns a DetectionResult for fixtures/sample-agent-app (unit test).
  • M2 — Provider bootstrap. src/agent/provider.ts — pickAgentModel + extension path resolution; the "ensure BitRouter reachable" flow. DoD: against a running local daemon, pickAgentModel returns a valid bitrouter/<id> (integration test may be skipped when no daemon, but code path is unit-tested with a mocked /models).
  • M3 — pi session + read-only tools + one emit. src/agent/session.ts, src/agent/tools/ (read/grep/find/ls + stub emit_bitrouter_yaml), src/programs/run-analysis.ts running just the YAML phase. DoD: running init against the fixture produces a validated bitrouter.yaml. (This is the vertical slice that de-risks the whole pi embedding — reach it early.)
  • M4 — Estimate engine. scripts/sync-registry.ts, src/pricing/, registry_price_lookup + emit_estimate, host-side blendedReductionPct. DoD: .bitrouter/analysis.json has a transparent rate card; math unit-tested.
  • M5 — Audit + full spine + UI + consent. emit_audit_report, all three phase prompts wired, WizardUI (functional + --yes), the consent gate. DoD: init on the fixture produces all three artifacts and the summary.
  • M6 — (next, not in v1 gate) watch/T1. Read bitrouter.db + /metrics, upgrade the estimate to observed dollars, propose a policy diff. Capability- gated; still zero-write.

19. Acceptance Criteria (v1 = done)

The build loop is complete when all of these hold on a clean tree:

  • pnpm build && pnpm typecheck && pnpm lint && pnpm test all pass.
  • node dist/bin.js --help lists init, audit, estimate; --version works.
  • Running init against fixtures/sample-agent-app produces exactly three new artifacts — bitrouter.yaml, bitrouter-audit.md, .bitrouter/analysis.json — and modifies/creates no other file in the fixture (assert the fixture git status shows only these, ignored via --out-dir to a temp dir in the test).
  • The generated bitrouter.yaml passes validation (§13.3).
  • bitrouter-audit.md lists every model id in the fixture with call-sites and call-classes, and a rate-card estimate with before/after $/Mtok and a blended %; no invented absolute dollar total.
  • No secret value appears in any artifact; all keys are ${VAR} placeholders.
  • package.json has no @anthropic-ai/claude-agent-sdk, magicast, or recast; the agent loop is pi (session.prompt).
  • The agent is given no write/edit/bash-to-source tool (grep the tool registry: only read/grep/find/ls + registry_price_lookup + emit_*).
  • A single consent disclosure is shown before analysis; --yes bypasses it non-interactively.
  • README.md documents install, the three commands, the artifacts, and the local/cloud targets.

20. Testing & Fixtures

  • fixtures/sample-agent-app/ — a tiny fake agentic repo: a package.json depending on openai + @anthropic-ai/sdk, and 2–3 source files with hardcoded model ids across obvious routine and complex call-sites (e.g. a cheap "summarize" call and an expensive "plan" call). This is the e2e target.
  • Unit tests (vitest): detection, pricing math (blendedReductionPct), yaml validation, prompt strings, tool path-safety (emit refuses paths outside --out-dir).
  • E2e smoke: run init --yes --out-dir <tmp> against the fixture with a mocked or live /models; assert the three artifacts + validation + the "no other files touched" invariant.
  • The pi/model calls in e2e may run against a live local BitRouter when present, otherwise a recorded/mock model response — do not hard-require network in unit tests.

21. Repo Conventions

  • Conventional commits; PR titles in the same format. Branch off main.
  • Keep README.md in step with the CLI surface.
  • No // @ts-ignore/eslint-disable to bypass checks; no dead code.
  • Secrets never logged or written.

22. References

  • pi docs: pi.dev/docs/latest (+ /sdk, /rpc, /json, /tui).
  • PostHog wizard (blueprint, MIT): github.com/PostHog/wizard.
  • @bitrouter/pi provider: github.com/bitrouter/bitrouter-pi-provider.
  • Sibling BitRouter checkout ../bitrouter: examples/bitrouter.yaml (yaml shape), registry/ (prices + model metadata), dist/schema/bitrouter.config.schema.json (validation), skills/bitrouter/ (the skill, also bundled by @bitrouter/pi), CLI.md (bitrouter config validate, start, route, observe status).

Appendix A — External Contracts (verify against installed versions)

pi SDK (~0.79) — createAgentSession({ model, cwd, sessionManager, resourceLoader, noTools, customTools }) → { session }; SessionManager.inMemory(cwd); DefaultResourceLoader({ cwd, additionalExtensionPaths, extensionFactories, systemPromptOverride, noContextFiles, noThemes, noPromptTemplates, noSkills, noExtensions }); session.bindExtensions({}); session.prompt(text) (resolves on a tool-less turn); session.subscribe(cb) (events: message_update{text_delta}, message_end, tool_execution_start|update|end, turn_start|end, agent_start|end); session.getSessionStats(); defineTool({ name, description, parameters: Type.Object({…}), execute: async (toolCallId, params) => ({ content: [{type:'text', text}], details }) }).

@bitrouter/pi — extension default-exports async (pi) => pi.registerProvider( "bitrouter", { baseUrl, api:"openai-completions", apiKey?, authHeader?, models }), discovering models from GET ${baseUrl}/models. Env: BITROUTER_TARGET (local|cloud), BITROUTER_BASE_URL, BITROUTER_API_KEY (brvk_, local).

BitRouter registry provider price shape (../bitrouter/registry/providers/*.yaml) — per model: input_tokens:{no_cache,cache_read}, output_tokens:{text} in $/Mtok. Model metadata (../bitrouter/registry/models/*.yaml): id, family, open_weights.