PLAN.md

modelschemas — Live AI Model Schema Service

Historical build log (phases 0–18 complete). New work is GitHub issues. Live map: CLAUDE.md.

A Cloudflare Workers service that gives agents live, validated access to AI model schemas (request/response JSON Schemas per provider endpoint, plus model metadata), with react-query-style server-side caching: persistent, stale-while-revalidate, auto-refreshed when providers ship new models or API revisions.

Borrows directly from TanStack AI PR #622 (@tanstack/ai-schemas):

• Provider registry pattern — a ProviderConfig per provider (fetch → load → activity-categorised merged specs). Providers: OpenAI, Anthropic, Gemini, xAI Grok, ElevenLabs, OpenRouter, FAL.
• Activity grouping — chat | image | video | audio | embeddings | moderation; platform/admin endpoints classify to null and are dropped.
• Endpoint-id-keyed schema maps with self-contained JSON Schemas ($ref closures bundled under $defs).
• Spec-source tricks — e.g. Anthropic's spec URL resolved from the Stainless .stats.yml; FAL's expand=openapi-3.0 param; Gemini Discovery→OpenAPI conversion.

Key difference from the PR: no codegen, no Zod, no npm package. We extract and bundle JSON Schemas at runtime (pure JSON manipulation — portable to Workers) and serve them over HTTP. Zod codegen stays in the npm package world; this service is the live API.

---

Loop protocol (completed)

This plan was executed by an autonomous loop. Each iteration:

1. Read this file. Find the first unchecked - [ ] task, top to bottom.
2. Implement exactly that task. Stay inside its scope; if you discover necessary side-work, add it as a new task in the right phase instead of doing it now.
3. Verify against the task's acceptance criteria. Every task that adds or changes behaviour must ship with tests covering it — write them as part of the task, not later. Run bun run test (plain bun run, not --bun — bun's ws shim can't start the cloudflare vitest pool), bun --bun run lint, and bun run typecheck; all must be green (lefthook enforces lint/format/typecheck on commit — never commit with --no-verify). For route/API tasks, exercise the endpoint against bun --bun run dev (vite cloudflare plugin provides local D1/KV bindings — note the plugin is skipped under vitest; Workers-runtime tests need @cloudflare/vitest-pool-workers, see task 0.3).
4. Check the task off (- [x]), append a one-line note under it if anything non-obvious happened, and commit with a conventional message (feat(api): ..., chore(db): ...).
5. Phase boundary: when the task just completed is the last task of its phase, run the full test suite and lint one more time — everything green is a hard gate before starting the next phase. Make a phase-closing commit (chore: complete phase N) even if the last task was already committed, and confirm CI (task 0.0) passes on that commit. Do not begin the next phase with a dirty tree, failing tests, or red CI.
6. If blocked (missing secret, upstream outage, ambiguous requirement), check the task off as - [x] ~BLOCKED: with the reason, move on to the next task, and surface the blocker in the iteration summary.
7. Stop the loop when every task is checked.

Rules:

• No task is done without tests: new behaviour gets new tests, and the whole suite passes before the task is checked off.
• Never edit src/routeTree.gen.ts by hand.
• All times stored as unix epoch integers; all ids lowercase kebab/slug.
• Every API response is JSON (errors included: { error: { code, message } }).
• No secrets in code. Local: .env.local / .dev.vars. Prod: wrangler secret put.

---

Architecture decisions (settled during the build)

• D1 is the source of truth, not KV. We need relational queries (models by provider/activity/capability), versioned schema history, diffs, auth tables, and webhook subscriptions — that's SQL. KV is the hot cache in front of D1 for schema blobs (read-heavy, immutable-per-version), keyed by content hash.
• Two-tier refresh loop (Cloudflare Cron Triggers):
  • Fast tier, every 15 min: poll each provider's cheap model-list endpoint (/v1/models etc.). New/removed model → update D1, record a change event, fire webhooks. This is what catches "new model just dropped".
  • Slow tier, daily 05:00 UTC: full OpenAPI spec sync per provider (port of the PR #622 fetch+merge+bundle pipeline, minus codegen). Content-hash each bundled endpoint schema; changed hash → new schema version row + change event.
• SWR semantics (the "server-side react query"): every cached entity carries fetchedAt + staleTime. Reads always serve immediately; if stale, a background revalidation runs via ctx.waitUntil. Cron is the backstop, SWR is the accelerator. Responses expose ETag, Last-Modified, and X-Stale-At so agents can build their own cache layers on top.
• Auth: Better Auth on D1 (drizzle adapter) with the agent-auth plugin (@better-auth/agent-auth) as the primary flow — the standard agent protocol: discovery at /.well-known/agent-configuration, POST /agent/register, capability grants, short-lived signed JWTs (replay-protected, audience-bound) verified in our routes via verifyAgentRequest. Capabilities are derived from our own openapi.json with createFromOpenAPI where possible, hand-declared otherwise. The api-key plugin stays as a low-friction alternative for simple bearer access. Anonymous read access allowed at a low rate limit; authenticated agents get higher limits + webhooks. Reads stay public — auth gates rates, subscriptions, and any future write surface, not schema access.
• Validation: @cfworker/json-schema (built for Workers; ajv's codegen trips CSP/ eval restrictions).
• Worker entry: wrangler.jsonc currently points at @tanstack/react-start/server-entry. Replace with a thin src/worker.ts that re-exports the Start fetch handler and adds scheduled (cron) — and later queue if we adopt Queues. Webhook delivery starts with waitUntil + a retry table drained by cron (no paid-plan Queues dependency).

---

Phase 0 — Platform foundation

• 0.0 CI pipeline. GitHub Actions workflow (.github/workflows/ci.yml) running on push to main and on pull requests: oven-sh/setup-bun, bun install --frozen-lockfile, bun --bun run check, bun --bun run lint, bun run typecheck, bun --bun run test, bun --bun run build. This is the gate the loop protocol's phase-boundary step checks against. Accepts: workflow file is valid (passes actionlint or a dry-run push) and the run is green on the commit that adds it.
  • Note: actionlint not installable in the sandbox; validated via YAML parse + all five commands green locally, then confirmed green on GitHub Actions (run 27326004002 on commit 320c2b0, the phase 0 closing commit). checkout bumped v4→v5 (node20 deprecation). Standing permission from Tom to push origin/main each iteration.
• 0.1 Move the database to D1. Add a d1_databases binding (DB) to wrangler.jsonc; switch src/db/index.ts to drizzle-orm/d1 reading the binding from the Cloudflare env (expose a getDb(env) helper — Workers have no module-scope env); update drizzle.config.ts to the d1-http driver for remote, keep local migrations working against wrangler's local D1 (bun run db:generate + wrangler d1 migrations apply DB --local). Remove better-sqlite3 and the demo todos table. Accepts: dev server boots, a trivial query against D1 works locally.
  • Note: dev server only boots under bun run dev (node runtime) — bun --bun run dev hangs before vite's ready banner (pre-existing, ws-related). Vite binds IPv6-only (http://[::1]:3000). db:migrate script now applies migrations to local D1 via wrangler; db:migrate:remote added. @cloudflare/workers-types added (type-only, drizzle optional peer).
• 0.2 Custom worker entry with cron support. Create src/worker.ts exporting { fetch } from the TanStack Start server entry plus a scheduled(controller, env, ctx) handler that dispatches on controller.cron. Point wrangler.jsonc main at it and declare two crons: */15 * * * * (models poll) and 0 5 * * * (spec sync). Accepts: wrangler dev --test-scheduled + curl /__scheduled?cron=*/15+*+*+*+* hits the handler; normal SSR routes still render.
  • Note: on wrangler 4.99 the test endpoint is /cdn-cgi/handler/scheduled?cron=… (/__scheduled 404s); verified against the built worker (wrangler dev --test-scheduled -c dist/server/wrangler.json) and under vite dev — both crons dispatch, SSR home renders in both. Entry-module named exports must be handlers only (workerd rejects exported constants), so cron strings stay module-local.
• 0.3 KV namespace + helpers. Add kv_namespaces binding (SCHEMA_CACHE). Write src/server/kv.ts with typed get/put wrapping JSON + content-hash keys. The cloudflare vite plugin is incompatible with vitest (it is skipped under VITEST in vite.config.ts), so set up @cloudflare/vitest-pool-workers (or a miniflare test harness) here for tests that need real KV/D1 bindings. Accepts: round-trip test passes under bun run test with a real KV binding.
  • Note: pool-workers 0.16 (vitest 4) replaced defineWorkersConfig with a cloudflareTest() vite plugin — config split into vitest.config.ts + unit/workers project files (*.worker.test.ts runs in workerd). Tests MUST run via plain bun run test: under --bun, bun's ws shim can't start the pool runner (hangs after "Timeout starting cloudflare-pool runner"); ci.yml, CLAUDE.md, and the loop protocol updated accordingly. Miniflare bindings are declared inline (not via wrangler.jsonc configPath) so the pool doesn't try to load the TanStack Start worker entry.
• 0.4 Better Auth on D1. Wire src/lib/auth.ts to the drizzle adapter over D1, add the agent-auth (@better-auth/agent-auth), api-key, and bearer plugins, generate auth tables into src/db/schema.ts via bunx --bun @better-auth/cli generate (agent-auth adds agent, host, grant, approval), migrate. Keep email/password enabled (it backs delegated-mode approvals and the dashboard later). Minimal agentAuth({...}) config for now — providerName/description, modes: ['autonomous', 'delegated'], empty capabilities (filled in Phase 5). Accepts: sign-up + session retrieval works through the existing /api/auth/$ catch-all locally; auth tables present in local D1.
  • Note: better-auth 1.6 moved apiKey to @better-auth/api-key (added as a dep); the option is providerDescription, not description. Auth is a createAuth(db) factory in src/lib/auth.ts (CLI-loadable via scripts/better-auth-config.ts); the runtime instance in src/server/auth.ts builds from cloudflare:workers env and passes tanstackStartCookies() via extraPlugins — that plugin can't be in the factory (vite-only #tanstack-router-entry imports break the workers test pool). Worker tests apply drizzle migrations via the pool's TEST_MIGRATIONS/applyD1Migrations pattern. Curl-verified sign-up + get-session (needs an Origin header, else 403 MISSING_OR_NULL_ORIGIN).

Phase 1 — Data model

• 1.1 Core tables in src/db/schema.ts (drizzle, sqlite dialect):
  • providers — id, displayName, specSourceUrl, modelsEndpoint, authEnvVar, lastPolledAt, lastSyncedAt, status.
  • models — id (slug), providerId, rawId, activity, displayName, contextWindow, maxOutput, modalities (json), pricing (json), capabilities (json), firstSeenAt, lastSeenAt, deprecatedAt nullable.
  • endpoints — id, providerId, activity, method, path, description.
  • schema_versions — id, endpointId, kind (input|output), contentHash, schema (json text, $defs-bundled), specRevision, createdAt, supersededAt nullable.
  • changes — id, type (model.added|model.removed|model.updated| schema.added|schema.updated|endpoint.added|endpoint.removed), providerId, subjectId, summary, payload (json diff), createdAt. This is the public changelog feed. Accepts: migration generated and applied locally; drizzle types compile.
  • Note: migration drizzle/0001_sweet_zeigeist.sql; activities/changeTypes exported as const arrays (drizzle text enums are type-level only). Worker tests cover insert + relational query across all five tables and FK enforcement.
• 1.2 Cache + delivery tables:
  • cache_meta — key, fetchedAt, staleTime, lastError, refreshing flag.
  • subscriptions — id, agentId (auth user), url, secret, events (json array), providerFilter nullable, active, createdAt.
  • webhook_deliveries — id, subscriptionId, changeId, attempt, nextAttemptAt, status (pending|ok|failed), lastResponseCode. Accepts: migration applies; FK relations defined.
  • Note: migration drizzle/0002_violet_lionheart.sql; deliveries indexed on (status, nextAttemptAt) for the cron drain. Worker tests cover cache_meta round-trip, subscription→user/delivery→change relations, and FK enforcement.
• 1.3 Seed script. scripts/seed.ts inserting the 7 providers with their spec source config (port the constants/URLs from PR #622's scripts/providers/*: Anthropic .stats.yml resolution, FAL expand=openapi-3.0 note, Gemini discovery URL, etc.). Runnable via bun run seed against local D1. Accepts: providers table populated locally.
  • Note: seed data lives in src/db/seed-providers.ts (typed $inferInsert, unit tested, reusable by the Phase 2 registry); scripts/seed.ts upserts via wrangler d1 execute --file (idempotent, preserves runtime columns; --remote flag for prod). URLs/env-vars ported from PR #622 @ 276e808.

Phase 2 — Ingestion pipeline (the refresh loop)

• 2.1 Provider registry. src/server/providers/types.ts + src/server/providers/{openai,anthropic,gemini,grok,elevenlabs,openrouter,fal}.ts. Port PR #622's ProviderConfig shape, adapted to Workers: fetchSpec(env) returns parsed spec objects (no filesystem), listModels(env) hits the provider's models endpoint, classify(path, op) → Activity | null. Providers needing keys are skipped with a recorded warning when the secret is absent. Accepts: unit tests for classify per provider using small spec fixtures.
  • Note: listModels also normalises to the ModelInfo shape (providers own their API mapping; 2.4 just diffs). FAL activity rides on a fetch-time per-operation marker (category-derived; non-taxonomy targets like 3d/json classify to null for now); OpenRouter's video synthesis + embeddings lift ported as exported, unit-tested helpers; Anthropic's resolved spec URL doubles as specRevision. yaml dep added for OpenAI/Anthropic YAML specs.
• 2.2 Schema extraction + bundling. src/server/ingest/bundle.ts: given a merged OpenAPI spec and an endpoint, extract the input schema (request body — handle application/json and multipart/form-data, per the PR's multipart fix) and output schema (post-200 strategy, plus FAL's sibling-get), then inline the $ref closure under $defs so every schema is self-contained. Stable stringify → SHA-256 content hash. Accepts: unit tests with fixture specs prove self-containment (no dangling $refs — port the PR's dedup-rename lesson as a test).
  • Note: content hashing reuses stableStringify/contentHash from src/server/kv.ts (task 0.3). Improvement over the PR: refs back to the root schema rewrite to # instead of dangling in $defs; findDanglingRefs exported as the self-containment checker (used by tests, reusable as a sync-time sanity check). Inline (ref-less) body schemas are bundled too, not skipped.
• 2.3 Sync engine. src/server/ingest/sync.ts: per provider — fetch spec → classify endpoints → bundle schemas → diff content hashes against schema_versions → insert new versions, mark superseded, upsert endpoints, write changes rows, warm KV with new blobs. Idempotent; per-provider try/catch so one provider's outage doesn't sink the run; record outcome on providers. Accepts: integration test with a fixture spec: first run inserts, second run no-ops, mutated fixture produces exactly one schema.updated change.
  • Note: endpoint db ids are ${providerId}/${path-minus-slash}. Vanished endpoints keep their rows + version history; an endpoint.removed change is written once (deduped against prior changes). Failed providers get status='degraded', successful syncs set lastSyncedAt + status='active'. Injectable clock for tests. D1 persists across tests in one isolate — worker tests use per-test provider ids.
• 2.4 Model poller. src/server/ingest/poll-models.ts: per provider, call listModels, normalise to the models shape, diff against D1 → model.added / model.removed / model.updated changes, bump lastSeenAt. Accepts: fixture test covering add/remove/no-change.
  • Note: normalisation lives in each provider's listModels (task 2.1); the poller diffs. "Removed" = deprecatedAt set (row kept), recorded once; reappearance clears deprecation via model.updated with a before/after payload. Model db ids are ${providerId}-${slugified rawId}.
• 2.5 Wire crons. scheduled dispatch: 15-min cron → pollAllModels(env, ctx), daily cron → syncAllSchemas(env, ctx). Stagger providers with ctx.waitUntil and keep each provider's work sequential to respect subrequest limits. Accepts: --test-scheduled run against live providers (no key required: OpenRouter) writes real rows locally.
  • Note: live run wrote 338 OpenRouter models + changes locally; second trigger idempotent (0 added); keyed providers skipped with recorded warnings. Surfaced a 0.4 regression: betterAuth() can't construct at module scope in the built worker (workerd forbids global-scope I/O/random) — src/server/auth.ts now lazy-inits via getAuth(). Use wrangler dev -c dist/server/wrangler.json --persist-to .wrangler/state so the built worker shares the root local D1.
• 2.6 Manual trigger + status. POST /v1/admin/sync/{provider} (admin-key gated) and GET /v1/status (public: per-provider lastPolledAt/lastSyncedAt/counts). Accepts: curl both locally.
  • Note: admin key via X-Admin-Key or Bearer (fails closed without ADMIN_KEY; local value in .env.local). Logic split into testable src/server/{admin,status}.ts. routeTree.gen.ts must be regenerated by the dev server, not tsr generate (the pinned router-cli 1.132 emits a format without the server route option). Live curl: status shows per-provider counts; admin sync of OpenRouter ingested 21 endpoints / 41 schema versions with zero dangling-ref warnings.

Phase 3 — SWR cache layer ("server-side react query")

• 3.1 src/server/cache.ts — swr(env, ctx, key, fetcher, { staleTime, hardTtl }): KV hit + fresh → return; KV hit + stale → return AND ctx.waitUntil(revalidate) guarded by the cache_meta.refreshing flag (cheap cross-isolate dedupe; a stuck flag older than 2× staleTime is ignored); miss → fetch inline, persist to KV + cache_meta. Accepts: unit tests for all three paths and the stuck-flag case.
  • Note: signature is swr(deps, key, fetcher, opts) with deps = {db, kv, waitUntil, now?} (injectable clock + collected waitUntil promises make the tests deterministic). Result carries fetchedAt/staleAt/revalidating for task 3.2's header helper. Failed background revalidation records cache_meta.lastError and keeps serving the stale value. 6 worker tests.
• 3.2 HTTP cache semantics. Response helper adding ETag (content hash), Last-Modified, Cache-Control: public, max-age=60, stale-while-revalidate=600, X-Fetched-At, X-Stale-At; honour If-None-Match → 304. Apply to all schema/model read endpoints in Phase 4. Accepts: test asserts 304 on matching ETag.
  • Note: cachedJson(request, value, opts) in src/server/http-cache.ts — etag computed from the body unless a stored content hash is passed; If-None-Match handles lists, weak validators, and *. X-Fetched-At/X-Stale-At are epoch seconds, matching SwrResult from 3.1.

Phase 4 — Public API (agent-first)

All under /v1, JSON-only, TanStack Start server-route handlers. Design for an LLM consumer: every list response includes _links to drill deeper, errors carry remedies.

• 4.1 Discovery surface. GET /v1 (service description + endpoint index), GET /llms.txt (markdown usage guide for agents), GET /openapi.json (the service's own OpenAPI doc, hand-maintained in src/server/openapi.ts). Accepts: all three render; openapi.json validates against OpenAPI 3.1 meta-schema in a test.
  • Note: meta-schema (2022-10-07) vendored in src/server/fixtures, validated with @cfworker/json-schema (added now, reused by 4.4); a second test enforces unique camelCase operationIds (the 5.1/7.4 contract). llms.txt content lives in src/server/llms-txt.ts for reuse by /docs (7.3) and the skill (7.6). Dotted filenames use [.] escaping (openapi[.]json.ts); the router plugin rewrites the createFileRoute id itself on dev-server regeneration.
• 4.2 Catalog endpoints.
  • GET /v1/providers — list with sync status.
  • GET /v1/providers/{provider}/models — models for one provider.
  • GET /v1/models?activity=&provider=&capability=&q= — cross-provider catalog, filterable; this is the headline "what can I use right now" endpoint.
  • GET /v1/models/{provider}/{modelId} — full metadata + _links.schemas. Accepts: curl each against seeded/synced local data; filters covered by tests.
  • Note: query logic in src/server/catalog.ts (worker-tested; routes thin). Deprecated models hidden unless ?deprecated=true; model detail resolves slug OR raw id; invalid ?activity= → 400 listing valid values. Curl-verified against live local data (q=claude → 20 OpenRouter models, capability=tools → 252, encoded raw-id lookup, 404s with remediation). All reads use the 3.2 cachedJson helper.
• 4.3 Schema endpoints.
  • GET /v1/schemas/{provider} — activities + endpoint ids.
  • GET /v1/schemas/{provider}/{activity} — endpoint-id-keyed map (current versions), mirroring the PR's endpoint-schema-map shape.
  • GET /v1/schemas/{provider}/{activity}/{endpointId}?kind=input|output&version= — single self-contained JSON Schema; defaults to current input schema. All via the SWR layer + ETag helper. Accepts: schemas served byte-identical to D1 content; 304s work; unknown ids → structured 404 with valid-id hints.
  • Note: endpoint ids contain slashes, so the single-schema route is a splat ($activity/$.ts); clients URL-encode (chat%2Fcompletions). waitUntil comes straight from cloudflare:workers (added to the local module declaration). Versioned (content-addressed) reads cache with staleTime 86400, current reads 300; single-schema ETag is the stored contentHash. Live-verified on synced OpenRouter data including a 304 round-trip; byte-identity asserted in worker tests.
• 4.4 Validation endpoint. POST /v1/validate with { provider, endpointId, kind?, payload } → { valid, errors: [{ path, message, keyword }] } using @cfworker/json-schema. Accepts: test with one valid + one invalid Anthropic messages payload.
  • Note: worker tests use a trimmed Anthropic messages fixture (valid + invalid with $defs-ref'd errors); also live-verified against the REAL synced Anthropic spec (keyless sync: 5 endpoints / 10 versions) — valid payload passes, bad payload yields structured required/enum errors. Response includes the schema contentHash so agents know which version judged them.
• 4.5 Changes feed. GET /v1/changes?since=&provider=&type= (cursor-paginated) — the polling-friendly alternative to webhooks. Accepts: cursor pagination test.
  • Note: opaque (createdAt, id) keyset cursor (base64), newest first, stable under concurrent inserts, tie-break tested on equal timestamps; limit clamped to 200; _links.next carries the ready-made next-page URL. Live-verified against real sync-generated change rows.

Phase 5 — Agent signup & access control

Built on Better Auth's agent-auth plugin (https://better-auth.com/docs/plugins/agent-auth): discovery → register → request capability grants → execute with short-lived JWTs.

• 5.1 Agent Auth provider surface. Expose the discovery document at /.well-known/agent-configuration (route handler calling auth.api.getAgentConfiguration()). Declare capabilities in agentAuth({...}): derive the read/validate operations from our own src/server/openapi.ts doc via createFromOpenAPI (every Phase 4 operation has an operationId), and hand-declare manage_subscriptions. Autonomous mode is the default path (no human in the loop); delegated mode + approvals kept enabled but not required for any current capability. Accepts: discovery doc renders; POST /agent/register → capability request → POST /capability/execute round-trips get_schema locally.
  • Note: full round-trip verified live by bun scripts/agent-roundtrip.ts (Ed25519 keys via jose; agent JWT: iss=hostId, sub=agentId, aud=default_location; execute response wraps results in {data}). Required allowDynamicHostRegistration: true (open signup) and a resolveAutonomousUser virtual identity (autonomous agents have no human owner). Admin op (syncProvider) filtered out of capabilities; capabilities are at /capability/list, not inline in the discovery doc.
• 5.2 JWT verification on owned routes. For agent-authed access to our native /v1/* routes (as opposed to the /capability/execute proxy), add a requireAgent(request) helper wrapping verifyAgentRequest(request, auth) + grant check, used by subscriptions (Phase 6) and /v1/agents/me. Accepts: test hits a protected route with a valid agent JWT (200) and without (401).
  • Note: signature is requireAgent(auth, request, {capability?}) returning a principal or a ready-made 401/403 Response. verifyAgentRequest builds its session URL without the basePath, so the helper retries with /api/auth appended. Agent JWT jtis are replay-protected — a fresh token per request. /v1/agents/me created minimal here (5.5 enriches it); 200/401/403 covered by worker tests + the live round-trip script.
• 5.3 API-key fallback. POST /v1/agents/register-key { name, email? } → Better Auth user + api-key plugin key, returned once; Authorization: Bearer <key> accepted everywhere requireAgent is — for agents that don't speak the agent-auth protocol. Accepts: register → authed call with the key succeeds.
  • Note: requireAgent now returns a Principal union (agent | api-key); non-JWT bearer values and X-Api-Key are treated as keys; api-key principals pass capability checks (full-access fallback). The plugin's key owner field is referenceId. Emails are pre-checked in D1 (better-auth's duplicate throw leaves a floating rejection). Live-verified: register → Bearer key on /v1/agents/me → 200.
• 5.4 Rate limiting. Per-agent/per-key and per-IP (anonymous) limits via Workers Rate Limiting binding (fallback: fixed-window counter in KV). Anonymous: 60 req/h; authenticated: 5k req/h. 429s include Retry-After. Accepts: test exhausts the anonymous window and sees 429.
  • Note: KV fixed-window used (the Workers binding can't express hourly windows); enforced centrally in src/worker.ts fetch for /v1/* (admin-keyed requests exempt). API keys are verified for their bucket; agent JWTs bucket by *decoded* sub — full verification would consume the replay-protected jti before the route's own auth. Eventual-consistency drift accepted (abuse mitigation, not billing). Acceptance test exhausts 60 anon requests → 429 with Retry-After; keyed bucket independent.
• 5.5 Usage endpoint. GET /v1/agents/me — agent identity, grants, limits, current usage. Accepts: curl with agent JWT and with API key.
  • Note: usage reads the live KV rate counter (non-incrementing readRateUsage). Curl-verified with both credential types. Also hardened 5.4 per security review: agent JWT buckets now require a signature+expiry check against the agent's stored key (jti untouched) — forged subs can no longer mint 5k/h windows (tested with rotating spoofed subs).

Phase 6 — Push notifications

• 6.1 Subscriptions CRUD. POST|GET|DELETE /v1/subscriptions (authed): url + events filter (+ optional provider filter). Secret generated server-side, returned once. Accepts: CRUD round-trip test.
  • Note: gated on the managesubscriptions capability; autonomous agents' virtual identities are materialised as user rows on first write (FK integrity); 10 subscriptions/owner cap; whsec-prefixed 32-byte secrets; list never echoes secrets; ownership enforced on delete. Live curl round-trip verified.
• 6.2 Webhook delivery. On new changes rows, enqueue webhook_deliveries; deliver via ctx.waitUntil with HMAC-SHA256 signature header (X-ModelSchemas-Signature), JSON body { event, change, _links }. Failures → exponential backoff via nextAttemptAt, drained by the 15-min cron; 8 failures → subscription auto-paused + recorded. Accepts: integration test with a local receiver route capturing the signed payload.
  • Note: fan-out runs off a cache_meta checkpoint (changes.createdAt high-water mark) so enqueue is idempotent without coupling sync/poll code; the 15-min cron runs runWebhookTick (enqueue + drain) inside the same waitUntil as the poll. Backoff is 60s·2^attempt; signature is sha256=<hex HMAC> over the exact body, verified by recomputation in the receiver test. Injectable fetcher = the "local receiver".

Phase 7 — Clients, MCP + DX polish

The service's own openapi.json (task 4.1, every operation has an operationId) is the single source for three consumer surfaces: the generated TS client, the CLI, and the agent-auth capability list (5.1). Spec drift breaks a CI check, not a user.

• 7.1 MCP server endpoint at /mcp (streamable HTTP, Workers-native) exposing tools: list_models, get_model, get_schema, validate_payload, recent_changes. Reuse the Phase 4 service functions directly. Accepts: MCP inspector (or a scripted client) lists tools and round-trips get_schema.
  • Note: hand-rolled stateless streamable-HTTP (JSON responses, no SSE session — the MCP SDK transports are node/DO-centric); handles initialize/ping/tools/*, 202s notifications, 405s GET, JSON-RPC errors for unknown methods. Live scripted client listed 5 tools and round-tripped the real Anthropic messages schema.
• 7.2 Minimal human dashboard. / route: provider sync status, model counts, latest changes, docs links. Tailwind + shadcn; no auth needed (read-only). Accepts: renders against local data.
  • Note: data via a createServerFn loader reusing getServiceStatus + listChanges (already worker-tested); plain Tailwind shadcn-style cards/badges (no new deps). Full SSR verified against local data (provider table + change feed in the HTML — note: ugrep chokes on the single-line SSR output, use python/sed to inspect).
• 7.3 Docs page /docs: quickstart for agents (register → fetch schema → validate → subscribe), rendered from the same content as llms.txt. Accepts: renders; links resolve.
  • Note: renders the llmsTxt module verbatim via streamdown (already a dep) — zero drift by construction; a content-contract test pins the sections/paths the page and the future skill depend on. All 7 nav links curl-resolved 200 live.
• 7.4 Generated TS client (@modelschemas/client). Convert the repo to bun workspaces (workspaces: ["packages/*"]); add packages/client generated by @hey-api/openapi-ts (pin the version — PR #622 pinned 0.97.2 for a reason) from our spec: add bun run openapi:emit (writes openapi.json from src/server/openapi.ts) and bun run generate:client. Fetch-based client, typed per operation, pluggable auth (API key header or agent JWT). CI-style check script: regenerate → git diff --exit-code so the committed client never drifts from the spec. Accepts: client compiles; smoke test against local dev lists providers and fetches one schema, fully typed.
  • Note: root devDependency @modelschemas/client: workspace:* makes bun link the package (workspaces alone don't symlink into the root). Generated code lives in packages/client/src/generated (eslint-ignored; typechecked by the root tsconfig); hand-written entry adds createModelschemasClient with bearer auth. check:client wired into CI. Smoke: bun scripts/client-smoke.ts listed 7 providers + fetched the Anthropic schema typed.
• 7.5 CLI (packages/cli, bin: modelschemas). Thin command layer over the generated client. modelschemas login runs agent-auth registration (autonomous mode by default; --delegated walks the device-authorization approval flow and polls until granted; --api-key falls back to /v1/agents/register-key), storing credentials in ~/.config/modelschemas/credentials.json (0600). Commands: models list|get, schema get <provider> <endpointId>, validate <endpointId> <file>, changes [--since], subscribe <url> [--events], whoami. --json everywhere (default when not a TTY) so the CLI is itself agent-friendly. Accepts: bun packages/cli login against local dev creates an agent; models list then succeeds authenticated; validate returns non-zero exit on invalid payload.
  • Note: all commands live-verified (login → active autonomous agent, 0600 creds, MODELSCHEMAS_CONFIG_DIR override for tests; whoami; models list/get; schema get auto-resolves activity from the provider index; validate exits 2 on invalid; changes; subscribe). validate takes <provider/endpointId>. Surfaced + fixed a 5.1 bug: defaultHostCapabilities derived from the raw spec included the excluded syncProvider op, 400-ing capability-less registrations — now the filtered list.
• 7.6 Distributable agent skill. skill/modelschemas/SKILL.md (frontmatter: name, description with trigger phrases like "model schema", "what models support X", "validate this provider payload") teaching the full workflow: discovery (/.well-known/agent-configuration, llms.txt) → auth via modelschemas login (or raw HTTP for keyless reads) → fetch/validate schemas → watch /v1/changes. Generate the endpoint reference section from the same source as llms.txt to avoid drift. Serve it: GET /skill returns the SKILL.md, linked from /docs and llms.txt. Accepts: frontmatter valid; installing the dir into .claude/skills/ and asking for an Anthropic chat schema produces a correct CLI/HTTP call sequence.
  • Note: SKILL.md generated from src/server/skill.ts (bun scripts/emit-skill.ts); a test pins the committed file to the module and validates frontmatter + trigger phrases; the reference section embeds llms.txt verbatim. Served at GET /skill, linked from /docs and llms.txt. The taught call sequences are the live-verified ones (schema get anthropic v1/messages; encoded HTTP path).

Phase 8 — Ship

• 8.1 Production setup. Create real D1 + KV resources, set IDs in wrangler.jsonc; wrangler secret put for BETTER_AUTH_SECRET, provider keys (FAL_KEY, ELEVENLABS_API_KEY, …), ADMIN_KEY. Document in README. Accepts: bun run deploy succeeds; /v1/status live.
  • Done 2026-06-11 after Tom's "deploy it": D1 557c12a0… + KV 8c8f93fc… created, IDs wired, remote migrations + seed applied, secrets set (BETTER_AUTH_SECRET, ADMIN_KEY — prod value in .env.local as PROD_ADMIN_KEY; provider keys still unset → those providers skip). Live at https://modelschemas.openstory.workers.dev with both crons. Deploy script fixed to use the vite-built config (wrangler deploy -c dist/server/wrangler.json); BETTER_AUTH_URL var set to the deployed origin.
• 8.2 First production sync. Trigger admin sync for all providers; verify /v1/models?activity=chat returns real multi-provider data in prod. Accepts: spot-check Anthropic + OpenRouter schemas served with ETags.
  • Done: admin sync of OpenRouter (21 endpoints/41 versions) + Anthropic (5/10) in prod, outcomes identical to local; /v1/models serves live multi-provider data (cron poll already fired); Anthropic messages schema served with the same contentHash as local, 304 on If-None-Match verified.
• 8.3 Observability. wrangler.jsonc observability.enabled, structured console.log JSON lines in cron handlers (provider, duration, changes count), README runbook for "a provider sync is failing". Accepts: logs visible via wrangler tail during a manual sync.
  • Note: observability.enabled set, worker renamed modelschemas, cron handlers already log structured JSON lines (provider/outcomes), README rewritten with the runbook + production setup. The wrangler-tail acceptance can only run post-deploy (8.1) — everything tail-able is in place. Post-deploy: wrangler tail captured both manual sync invocations (request events, outcome ok).
• BLOCKED 8.4 Publish client + CLI. npm publish @modelschemas/client and the modelschemas CLI (changesets or manual version bump); install instructions in README, /docs, and the skill. Accepts: bunx modelschemas@latest whoami works against prod.
  • BLOCKED 2026-06-11: npm publish is irreversible and needs Tom's npm account + scope decision (@modelschemas org availability unverified) + a deployed prod URL (8.1). Packages are publish-shaped (files/bin/exports set).
  • 2026-08-21 (#48): scope kept as @modelschemas (CLI unscoped). 0.1.0 could not go through Actions (npm trust / npm stage need an existing package). Laptop bootstrap landed 2026-08-21 (org, npm publish, stage-only npm trust, tokens disallowed). Ongoing: OIDC npm stage publish from .github/workflows/publish.yml (no NPM_TOKEN; provenance on future Actions stages).
• 8.5 Update CLAUDE.md with the new architecture (D1/KV bindings, cron entry point, ingest pipeline layout, workspace packages, API map) replacing the starter description. Accepts: CLAUDE.md matches reality.
  • Note: full rewrite — commands (bun/--bun split, db/codegen scripts), worker entry
    • ingest/API/auth/webhook architecture, workspace packages, route-tree-regen and module-scope-auth gotchas, API map. README rewritten alongside (8.1 docs + 8.3 runbook).

Phase 9 — Human users + site polish

Requested by Tom 2026-06-11: a human-facing surface alongside the agent one. The production domain is modelschemas.com.

• 9.1 Custom domain. Attach modelschemas.com to the deployed worker (Cloudflare custom domain on this account's zone; routes in wrangler.jsonc with custom_domain: true), set the BETTER_AUTH_URL var to https://modelschemas.com (agent JWT audiences are origin-bound), redeploy, and confirm the workers.dev URL still serves (or document the canonical-host choice). Accepts: https://modelschemas.com/v1/status live. If the zone is not in the OpenStory account, mark BLOCKED with what Tom needs to do.
  • First attempt 2026-06-11 failed: the modelschemas.com zone is not in the OpenStory Cloudflare account (PUT …/domains/records rejected). Prereq for Tom: add modelschemas.com as a zone (dashboard → Add site → point registrar nameservers at Cloudflare). The loop should retry this task first each iteration and skip forward while the zone is absent. Route was reverted so deploys stay green; service stays on modelschemas.openstory.workers.dev meanwhile. NOTE: 9.2's email sender verification ALSO depends on this zone.
• 9.2 Email OTP sign-in for humans. Add better-auth's emailOTP plugin to createAuth with a mail abstraction in src/server/email.ts built on Cloudflare Email Workers — a send_email binding (EMAIL) in wrangler.jsonc using the structured env.EMAIL.send({to, from, subject, text}) API, sender login@modelschemas.com. Until the zone is verified (9.1), and in local dev, every send falls back to a structured console line ({"job":"email",...}) so the OTP stays observable — never silently drop a sign-in. The sender is injectable so worker tests capture OTPs. Build a minimal /login page (email → code → session) with the email-otp client plugin, plus a signed-in indicator + sign-out. Keep email/password enabled (delegated-mode approvals depend on it). Regenerate worker-configuration.d.ts after adding the binding. Accepts: worker test issues + verifies an OTP via an injected sender; live dev sign-in works end to end using the logged code.
  • Note: live-verified — the vite plugin SIMULATES the send_email binding locally and writes messages to a temp .txt (path printed in dev output), so the dev fallback log rarely fires; the OTP was read from the simulator file and the full send→sign-in→session flow passed. better-auth's withSpan telemetry leaves a floating APIError rejection on failed sign-ins — filtered via vitest's root onUnhandledError (APIError only). Production email needs Email Routing enabled on the modelschemas.com zone (sender verification) — covered in 9.5.
• 9.3 Human API-key management. /account page for signed-in users: list, create (name + optional expiry), and revoke their API keys via the better-auth api-key plugin's session-authed endpoints; the key value is shown exactly once on creation, with copy-paste usage snippet (Authorization: Bearer …). The page explains what keys unlock (5k req/h + subscriptions). Accepts: live flow — sign in, create key, key authenticates /v1/agents/me, revoke, key 401s; worker tests for the session-scoped key lifecycle.
  • Note: list endpoint returns {apiKeys: []}, not a bare array. Live flow verified end to end (OTP sign-in → session-cookie key create → key 200s /v1/agents/me → revoke → 401); worker tests cover the lifecycle plus cross-user isolation (Bob can't see/revoke Alice's key).
• 9.4 Frontend overhaul (use the /frontend-design:frontend-design skill). Make the site look hot while staying agent-first: redesign the landing page (/) with a hero that pitches the service, a prominent "for agents" section (copy-paste curl + CLI + MCP + skill install examples, llms.txt/discovery links), the live provider status + latest-changes data, and nav to /docs, /login, /account. Restyle /docs, /login, /account to match. No regression to any agent surface (JSON endpoints untouched). Accepts: pages render with the new design against local data; all links resolve; existing tests stay green.
  • Note: direction = "machine registry" — dark instrument panel (phosphor green / amber on near-black), Instrument Serif display + IBM Plex Mono data + Instrument Sans body, terminal-chrome cards, pulsing status dots, scanline overlay, staggered hero reveal (reduced-motion safe). Shared chrome in src/components/site.tsx; landing hero features a real schema fragment + etag, a 4-card FOR AGENTS section (curl/validate/MCP/discovery+skill), live provider table and log-line change feed. All 9 routes/surfaces curl-verified; agent JSON untouched; 128 tests green.
• 9.5 Ship the human surface. Deploy to production and verify on the live origin (modelschemas.com once 9.1 lands, else workers.dev): OTP sign-in (real email via the sendemail binding when the zone is verified; logged code via wrangler tail otherwise), key creation, the key authenticating /v1/agents/me, and the redesigned landing page rendering. _Accepts: live production walkthrough recorded in the note.

Phase 10 — Agent-readiness audit (isitagentready.com)

Tom pasted the audit 2026-06-11. Fixes target https://modelschemas.com; each lands as a route/header served by the worker so they stay correct on every deploy. Re-verify on production at the end.

• 10.1 Sitemap + robots. GET /sitemap.xml (canonical pages: /, /docs, /login, /account, plus /llms.txt, /openapi.json, /skill), generated dynamically (origin from BETTERAUTH_URL, lastmod from the latest change row) so it is always current on publish; GET /robots.txt allowing all and referencing the sitemap. _Accepts: both render locally with correct content types; sitemap validates against the sitemaps.org shape (urlset/url/loc).
  • Note: a scaffold public/robots.txt static asset was shadowing the route (assets win over worker routes) — removed. lastmod derives from the newest change row; worker tests cover both surfaces.
• 10.2 Link headers (RFC 8288). Add Link response headers on HTML pages (via the worker fetch wrapper): </.well-known/api-catalog>; rel="api-catalog", </openapi.json>; rel="service-desc", </docs>; rel="service-doc", </llms.txt>; rel="describedby". Accepts: curl -I of / shows the Link header; JSON endpoints unaffected.
• 10.3 Discovery trio. /.well-known/api-catalog (application/linkset+json per RFC 9727: anchor + service-desc/ service-doc/status), /.well-known/mcp/server-card.json (SEP-1649: serverInfo, transport endpoint /mcp, capabilities/tools), and /.well-known/agent-skills/index.json (Agent Skills Discovery RFC v0.2.0: $schema + skills[] with name/type/description/url/sha256 — the digest computed from the live SKILL.md content). Accepts: all three render with correct content types; sha256 in the index matches the served /skill body.
• 10.4 Auth discovery. /.well-known/oauth-protected-resource (RFC 9728: resource, authorizationservers, scopes_supported = public capability names), /.well-known/oauth-authorization-server (RFC 8414 fields we genuinely have: issuer, registration via agent-auth, the device-authorization endpoint, jwks where applicable) including the auth.md agent_auth block (register_uri, identity/credential types), and /auth.md — markdown registration instructions for agents (agent-auth protocol + register-key fallback), generated from the same sources as llms.txt where possible. Honest metadata only — no fake OAuth endpoints. _Accepts: all three render; agent_auth.register_uri round-trips against the real register endpoint locally.
• 10.5 Markdown negotiation. Requests with Accept: text/markdown on HTML pages return markdown with Content-Type: text/markdown (worker wrapper): /docs → llms.txt content, /skill already markdown, / → a markdown landing summary (hero pitch + agent quickstart + live links); browsers keep HTML. The audit saw a 500 — must return 200. Accepts: curl with the Accept header gets markdown 200s on /, /docs; plain requests still get HTML.
• 10.6 WebMCP. On the landing page, feature-detect navigator.modelContext and provideContext() with tools wrapping the public API: list_models, get_schema, validate_payload (fetch wrappers with JSON Schema inputs). No-op when the API is absent. Accepts: guarded registration code ships on / without console errors in normal browsers; tool definitions match the MCP tool shapes.
• 10.7 DNS-AID (+ ship + reverify). Attempt to publish _index._agents.modelschemas.com ServiceMode HTTPS/SVCB records (alpn h2, target the apex) via the Cloudflare API; DNSSEC needs zone signing + a DS record at the registrar — document for Tom if the API token lacks DNS scope. Then deploy everything and re-verify each Phase 10 surface on https://modelschemas.com. Accepts: production curls for 10.1–10.5 all pass; DNS-AID either resolves or is marked BLOCKED with exact records for Tom. Deployed + reverified 2026-06-11: production curls for 10.1–10.5 all pass (sitemap/robots, Link header HTML-only, discovery trio with sha256 match, auth trio with live register_uri, markdown negotiation). DNS-AID: BLOCKED on Tom — wrangler's OAuth token has no DNS scope (and harvesting it was denied anyway). To publish, in the Cloudflare dashboard (zone modelschemas.com → DNS) add an HTTPS record: name _index._agents, priority 1, target modelschemas.com., params alpn="h2" (zone-file form: _index._agents 3600 IN HTTPS 1 modelschemas.com. alpn="h2"), then enable DNSSEC (DNS → Settings) and add the DS record at the registrar.

Phase 11 — Agents API ergonomics (Tom, 2026-06-11)

Agents consume APIs differently from programmers. Keep the DX surface (/openapi.json, typed client) separate from the agent experience (AX) surface; these tasks build the AX layer.

• 11.1 HAL self-description. Add _links (HAL) to /v1 and every /v1/* JSON read response, extended per link with method, contentType, and an inline example (HAL doesn't carry these — agents need them). Keep payload shapes backwards-compatible (additive only); document the extension in llms.txt. Accepts: GET /v1 and GET /v1/models responses carry _links whose hrefs all resolve 200 locally (worker test walks them); openapi.json regenerated without breaking check:client.
• 11.2 Narrative API root. Make a JSON-accepting GET / (Accept: application/json, plus the GET /v1 self link) return the equivalent of MCP default instructions: narrative description, workflow, inline examples, and HAL links — the same content an MCP initialize instructions field would carry, generated from the llms.txt source. Accepts: JSON GET / returns the narrative document with _links; HTML browsers still get the landing page; MCP initialize instructions and the root narrative share one source.
• 11.3 ?wait= long-poll. Support ?wait=<seconds> (cap 60s, also accepts 60s form) on /v1/changes and /v1/status: hold the request until new data exists or the window expires (poll D1 every ~2s), returning whatever is current at expiry. Agents rarely have a sleep tool — this gives them one. Document on the endpoints + llms.txt; ensure Workers runtime limits are respected (use ctx.waitUntil-safe timers within the request, no cron dependency). Accepts: worker test proves ?wait=2 returns early when a change lands mid-wait and at ~wait expiry otherwise; rate limiter counts the call once.

Phase 12 — Build-time schema pulls: typegen + vite plugin (Tom, 2026-06-12)

Design settled with Tom (don't relitigate):

• Generated files on disk, committed. No virtual modules. Dev-time fetch only: the vite dev server (or explicit CLI command) fetches and writes; production builds touch zero network and fail with remediation if files are missing. .manifest.json in the out dir is the lockfile (endpoint → contentHash) and records the selection, so update needs no separate config.

• Tree-shakeable, no barrel index. One pure .ts module per endpoint+kind (schema const + generated types), zero imports, zero side effects, descriptive export names (anthropicV1MessagesRequestSchema, AnthropicV1MessagesRequest). Users import directly from the file. The schema const is annotated with an exported structural JsonSchema alias (exported so declaration: true consumers don't hit TS4025) — never as const (literal-typing megabyte schemas kills tsc).

• Server-side typegen, dependency-free emitter. No json-schema-to-typescript (prettier dep + bundle weight in workerd); we own a small deterministic emitter. Generated lazily at request time via ?format=types on the existing schema route, SWR-cached, ETag = contentHash + emitter version. tsconfig variance handled by emitting flag-agnostic declaration-only types plus one knob: optional=exact|undefined (default exact: foo?: T — JSON has no undefined). Always unknown, never any. Index signatures only when additionalProperties is explicitly truthy (jstt's always-on [k: string]: unknown poisons DX).

• Formatting belongs to the user. The pull core runs the user's own prettier (dynamic import('prettier') + resolveConfig) over generated files when present; otherwise the server's canonical formatting stands. Files carry a // @generated banner (source URL + contentHash + emitter version, no timestamp — output stays byte-deterministic).

• Selection grammar maps 1:1 onto the API: provider[/endpointId][#request|response] with * globs expanded via the catalog (request/response are the user-facing aliases for kind=input/output; filenames use them too: anthropic/v1-messages.request.ts).

• Auth: anonymous fine (a dev boot is ~1 request thanks to the manifest); MODELSCHEMAS_API_KEY lifts to the credentialed tier.

• Packages: @modelschemas/codegen (pull core: grammar, catalog expansion, ETag fetch, manifest, file writer, prettier pass), @modelschemas/vite (thin plugin), CLI gains pull/update on the same core.

• 12.1 TypeScript emitter. src/server/typegen.ts: JSON Schema → self-contained .ts module text (banner, JsonSchema alias, schema const, root interface/type, named $defs as exported declarations, JSDoc from descriptions). Handles type arrays, OpenAPI nullable, enum/const, anyOf/oneOf unions, allOf intersections, tuples, $ref → #/$defs/* (circular-safe via named declarations), name collisions via numeric suffix. Deterministic output; TYPEGEN_VERSION exported for ETags. Accepts: unit tests cover the subset + a real bundled fixture; generated text compiles clean under strict with exactOptionalPropertyTypes both on and off (in-memory tsc program in the test).

• 12.2 ?format=types on the schema route. Extend /v1/schemas/{p}/{activity}/{endpointId} with format=json|types (+ optional=exact|undefined): types responses are text/typescript; charset=utf-8, SWR-cached per contentHash+TYPEGEN_VERSION+optional, ETag/304 preserved. openapi.json + generated client regenerated (additive). Documented in llms.txt. Accepts: worker test fetches types for a synced fixture endpoint, gets compilable TS, 304 on If-None-Match replay; check:client passes.

  • Note: deviated from "SWR-cached emitted text" — the module is emitted inline from the already-SWR-cached schema (pure CPU, cheaper than a second KV read); the ETag (typesEtag = hash+TYPEGEN_VERSION+optional) still fully keys the body. Verified live: real Anthropic v1/messages → 186KB module, 0 tsc diagnostics under strict+exactOptionalPropertyTypes, 304 replay, 400s on bad format/optional. SKILL.md re-emitted (it embeds llms.txt).

• 12.3 @modelschemas/codegen pull core. packages/codegen: selection parsing/expansion, conditional fetch (If-None-Match from manifest), atomic file writes, manifest read/write, optional local prettier pass, pull()/checkUpdates()/verify(). checkUpdates is one /v1/changes?since= call per provider filtered to selected endpoints. Accepts: unit tests with a mocked fetch cover grammar (provider, globs, #kind), 304 short-circuit, manifest round-trip, verify failure messaging.

• 12.4 @modelschemas/vite plugin. packages/vite: on serve, pull missing files then log (never auto-rewrite) upstream updates from checkUpdates; on build, verify() only — zero network — erroring with "run modelschemas pull" remediation. Accepts: unit tests drive the plugin hooks both modes; a fixture project under the test proves generated modules are import-clean (no side effects, no barrel).

• 12.5 CLI pull/update. modelschemas pull taking selections plus --out and --no-types flags, and modelschemas update --out (selection read from the manifest) on the codegen core; help text updated. Accepts: unit tests for arg parsing; roundtrip exercised in 12.6.

• 12.6 E2E + docs + ship. bun scripts/pull-roundtrip.ts: against the local dev server, pull a real selection, typecheck the generated files, mutate one upstream schema (admin sync fixture or direct D1 write), prove update rewrites exactly that file. README + docs page + llms.txt get the vite-plugin/CLI quickstart. Deploy; verify ?format=types on production. Accepts: roundtrip script green locally; production curl returns compilable TS with correct content-type and ETag.

  • Note: roundtrip green against local dev (pull → tsc-clean module → 304 no-op → staled manifest entry rewritten exactly); upstream mutation done by staling the lockfile entry rather than a D1 write — same code path, no SWR-staleness flake. CLI pull/update smoked end-to-end. Docs: README quickstart; /docs + llms.txt already carry the 12.2 section. Deployed + verified 2026-06-12 (Tom approved): production curl → 200 text/typescript with <hash>-types-v1-exact ETag (and the optional=undefined variant keyed separately); full bun scripts/pull-roundtrip.ts https://modelschemas.com green — pull → tsc-clean → 304 no-op → staled-entry rewrite, all against prod.

• BLOCKED on Tom 12.7 Deploy on push via Workers Builds. Tom chose Cloudflare Workers Builds over a CI deploy job (no tokens in GitHub). Connecting the repo needs interactive GitHub auth in the Cloudflare dashboard, so the setup is Tom's: dashboard → Workers & Pages → modelschemas → Settings → Build → Connect → GitHub repo modelschemas/modelschemas (moved from tombeckenham/modelschemas 2026-06), production branch main, then:

  • Build command: bun run build (the build image auto-runs bun install from bun.lock)
  • Deploy command: bunx wrangler d1 migrations apply DB --remote && bunx wrangler deploy -c dist/server/wrangler.json (migrations are idempotent; the emitted config under dist/server is the real worker config — NOT the repo-root wrangler.jsonc)
  • Root directory: /; leave non-production branch builds enabled if PR preview versions are wanted (wrangler versions upload). Once connected, manual bun run deploy becomes the emergency path only. Accepts: a push to main produces a dashboard build that deploys; /v1/status on prod reflects the new version.

Phase 13 — Backdate firstSeenAt to real release dates (Tom, 2026-07-14; issue #1)

• 13.1 Backdate models.firstSeenAt per provider. ModelInfo gains optional releasedAt (epoch s); listModels populates it — OpenAI/xAI/ OpenRouter created, Anthropic created_at (ISO), FAL metadata.date (ISO), Gemini/ElevenLabs from a curated map in src/server/providers/release-dates.ts (+ Gemini MM-YYYY preview-id suffix fallback). The 15-min poll applies it continuously: on insert firstSeenAt = releasedAt when sane-and-earlier than now; existing rows backdate in place when upstream reports an earlier date — SILENTLY (no model.updated event: the fleet-wide first pass would flood the changes feed + webhook fan-out; converges once per row so the bulk clean-path perf note above still holds). Backdates write in 30-row CASE-chunked bulk UPDATEs — each D1 query is a subrequest against the invocation's 1,000 budget, and the ~1,900-row first pass as one-per-row writes would exhaust it mid-poll (and starve the webhook drain sharing the invocation). Never forward-dates; bogus timestamps (< 2015-01-01) ignored. PollOutcome.backdated counts corrections for cron observability. Accepts: live listModels shows date coverage openai 129/129, anthropic 10/10, grok 10/10, openrouter 344/344, fal 1399/1399; worker test walks insert/backdate/ never-forward/update-compose cycles.

Phase 14 — BytePlus provider (2026-08-11)

• 14.1 BytePlus as the 8th provider — fully static, keyless. BytePlus publishes no machine-readable spec (its OpenAPI docs live behind the console's API explorer) and the Ark data-plane GET /models needs a region-bound key AND is documented non-exhaustive, so both surfaces are embedded, ported from TanStack AI's @tanstack/ai-byteplus adapter (whose wire types are hand-written, test-pinned, and probe-verified against the ap-southeast host on 2026-07-31): src/server/providers/byteplus-spec.ts synthesizes two OpenAPI 3.1 documents — Ark (/chat/completions OpenAI-compatible + thinking/ reasoningeffort/repetition_penalty/service_tier extensions, /images/generations Seedream, /contents/generations/tasks Seedance async task API whose 200 is just the {id} ack, post-200 precedent per OpenRouter/ElevenLabs job acks) and Seed Speech (/tts/create, /auc/bigmodel/recognize/flash — separate host and X-Api-Key product key). byteplus.ts classifies by exact path and serves the curated 32-model catalog (18 chat / 7 video / 5 image / 2 speech) with context windows, modalities, probed structured-output capabilities, and releasedAt from the -YYMMDD id suffix (byteplusIdSuffixDate in release-dates.ts). Provenance: sources point at the human docs, hashes are contentHash of the built documents (FAL embedded-spec precedent), so content changes only land through edits here and version like any upstream change. NOTE the HMAC BYTEPLUS_ACCESS_KEY/SECRET_KEY pair in the old providers-to-add table only guards the control plane — never needed. _Accepts: byteplus.test.ts walks classifyAndBundle over the embedded specs (5 endpoints, zero warnings, no dangling refs, deterministic hashes) and the catalog counts; registry/seed tests updated to 8.

Phase 15 — BytePlus spec generated from the Go SDK (2026-08-12)

• 15.1 Generate the Ark spec from BytePlus's Go SDK at sync time. Probing the Ark data plane with a real ARK_API_KEY confirmed there is no fetchable OpenAPI document (/openapi.json, /swagger.json, /docs, /models/{id} all answer 200 with an EMPTY body). But BytePlus's official Go SDK declares the whole Ark data plane as json:"…"-tagged structs, and — unlike the Python SDK's Pydantic models, which need a Python runtime to introspect — Go structs are plain text. So go-struct-parser.ts (Workers-safe, pure string parsing) turns them into component schemas and byteplus-ark-build.ts maps them onto paths, letting fetchSpec regenerate the Ark document from raw.githubusercontent.com/byteplus-sdk/byteplus-go-sdk-v2 on the daily cron. No codegen step, no committed artifact, no CI drift check — upstream SDK releases flow in and the content-hash diff versions them like any other provider's spec change. Two deliberate seams: (a) the endpoint→type map is hand-maintained, because the model package has no notion of HTTP paths and paths change far more slowly than field sets; (b) an explicit curated overlay re-applies the request fields the SDK omits (reasoning_effort, top_k, seed on chat; stream on images; output_format on video) and the hand-written descriptions carrying probe findings that a type declaration cannot state. The overlay is an explicit list rather than a blanket merge of the old document, so a field genuinely REMOVED upstream disappears instead of being resurrected forever. Falls back to the embedded document (with a warning on the sync outcome, via the new SpecFetchResult.warnings) when GitHub is unreachable or the SDK outgrows the parser — degraded freshness, not a degraded service. Seed Speech stays hand-written: it appears in no BytePlus SDK in any language. Accepts: generated-vs-hand-port diff loses ZERO fields and gains three (function_call, image-response tools, task-response safety_identifier); live end-to-end against GitHub bundles all 5 endpoints with no warnings and no dangling refs; parser unit tests cover primitives/pointers/omitempty-requiredness, const-block enums, slices/maps/interface{}/[]byte, json:"-" and embedded fields, aliases and named composites, doc comments, closure-only emission, and graceful degradation on unmapped types.

• 15.2 Per-endpoint schema provenance (derivation + verifiedAt). Surfaces how each schema's content was arrived at so agents can weigh how far to trust it — and so a hand-written corner going stale is visible rather than invisible. A four-rung trust ladder on provenance.derivation: upstream-spec (extracted from a document the provider publishes and we re-fetch every sync — the seven original providers), generated (re-derived each sync from a provider artifact that is not a spec — BytePlus Ark from the Go SDK), probe-verified (hand-written but confirmed against live calls on verifiedAt — BytePlus TTS, 2026-08-12) and docs-derived (hand-written, never confirmed — BytePlus ASR, blocked on a Seed ASR account grant). The first two self-heal; the last two are point-in-time claims. Providers declare defaultDerivation, and any operation may override it with an x-modelschemas-provenance marker (the FAL activity-marker trick) — which is how the BytePlus Ark document downgrades itself from generated to probe-verified/2026-07-31 when it falls back to the embedded copy. A malformed marker falls back to the provider default rather than throwing. Stored on schema_versions beside the existing sourceUrl/sourceHash/extractorVersion provenance (nullable, additive migration 0004), so it versions per schema version and can improve over time as endpoints get probed. Accepts: worker tests assert the columns persist and the API surfaces them (including null for rows synced before the field existed); byteplus tests pin all five endpoint grades and the fallback downgrade; openapi.json, the typed client and SKILL.md regenerated.

Phase 16 — Full provider roster + system-wide schema discovery (2026-08-14)

• 16.1 Registry is the source of truth for provider listings. The landing page and /v1/status//v1/providers only showed providers whose rows happened to exist in D1, and the sync/poll engines only ever UPDATE provider rows — so the adapter providers added in #18–#37 never appeared in prod without a manual bun run seed. Two fixes, both self-healing: getServiceStatus merges providerRegistry (code) over the DB rows, surfacing never-synced providers as a new pending status with zero counts (blue dot on the site; enum added to the OpenAPI ServiceStatus schema); and ensureProviderRow (sync.ts) upserts the seed row — seedForProvider: hand-written row for the original 8, seedFromAdapter otherwise, no-op for configs without a specSourceUrl (test stubs) — at the top of every syncProvider/ pollProviderModels run, matching scripts/seed.ts semantics (config columns refresh, runtime state preserved). Accepts: status.worker.test asserts every registry provider surfaces as pending on an unseeded DB; sync/poll worker tests unchanged.

• 16.2 GET /v1/schemas — the system-wide schema index. getSystemSchemaIndex groups every endpoint into provider → activity → endpoint-id lists (one D1 query, SWR-cached under schema-index:all, ETag like every read). Discovery wiring: _links.schemaIndex on GET /v1, an /v1/schemas path in openapi.ts (client + SKILL.md regenerated), llms.txt quickstart step 3, and a list_schemas MCP tool (optional provider arg narrows to the per-provider index) mirrored into the WebMCP tool list. llms.txt no longer hard-codes the original 7-provider list.

• 16.3 /schemas — the human-browsable index. New route modeled on /models: the flattened endpoint list with provider/activity/q filters (client-side; the whole index is one loader payload), capped at 300 rows with a pointer to the API, readable/json ViewToggle serving the /v1/schemas body verbatim. Nav gains a schemas link; the landing hero gains a GET /v1/schemas button; provider-table schema counts link to /schemas?provider=….

• 16.4 schema-studio browses every schema live. The example keeps its four vite-plugin-pinned, offline-verified schemas as instant entries and adds a live rail: fetch /v1/schemas on mount, pick a provider, fetch any input schema on demand and render the generated form. Same-origin on the zone; from a local dev server it hits prod — which works because the worker now serves CORS-open reads (Access-Control-Allow-Origin: * + OPTIONS preflight) on GET/HEAD for /v1/* (admin excluded), /openapi.json and /llms.txt — a public keyless JSON API browsers can actually call.

Phase 17 — sync scalability (the FAL degradation)

• 17.1 Root-cause FAL's stuck degraded status. Production D1 showed the daily 05:00 spec sync dying mid-FAL every day since Aug 1: fal + the four biggest core providers flapping degraded, and every provider after fal in the registry (byteplus + all adapters) with last_synced_at NULL — never synced once. Cause: the sync did 2 D1 findFirst round trips per endpoint (+1 per vanished endpoint, forever) and FAL is ~1,400 endpoints — past the invocation's 1,000-subrequest budget and most of its wall clock, so a full successful sync (the only thing that resets status to active) could never complete. Fix mirrors poll-models' batching: one joined query for the provider's current versions, one for the endpoint.removed set; degraded-status write is best-effort so budget exhaustion inside the catch can't sink the rest of the run. FAL's paginated crawl also now retries 5xx/408/timeouts/network errors (not just 429), honors Retry-After, 5 attempts.
• 17.2 Shard the spec-sync cron. One shared invocation meant one provider's blowup starved everyone behind it (and uncatchable CPU/memory kills took out the rest of the registry silently). The daily sync now fires four crons (05:00/05:10/05:20/05:30 UTC), each a fresh invocation with its own budgets: shard 0 is FAL alone, the rest round-robin (specSyncShard; keep SPEC_SYNC_SHARD_CRONS in lockstep with wrangler.jsonc — a unit test enforces it). limits.cpu_ms raised to 300s (ceiling, not floor) for FAL's 1,400-spec parse+hash. Models poll unchanged: it's already batched, time-sensitive (15-min freshness), and carries the webhook drain.

Phase 18 — grain=provider model activity + route binding (issue #50)

• 18.1 Activity + displayName on grain=provider catalog rows. Grok and OpenAI /models payloads are id + created only, so every row listed activity: null and GET /v1/models?provider=grok&activity=image returned empty. Derive activity from the raw id (Grok Imagine image/video/voice vs chat; OpenAI gpt-image/sora/whisper/tts before the gpt- chat default) and a display name from the id. Anthropic listed models are chat. Persisted by the 15-min poller like Gemini already was.
• 18.2 Bind each model to its generation route schema. Catalog rows gain schemaEndpointId (and _links.schema) pointing at the shared HTTP route — v1/images/generations for Grok Imagine image, images/generations for gpt-image-2, v1/messages for Anthropic, Gemini's :predict / :generateContent / :predictLongRunning. Model-grained FAL falls back to the raw id. GET /v1/schemas/{p}/{activity}/{rawId} aliases onto that route and pins the request model field to the one id so a Run form can be built the same way as for FAL. Validate accepts the same alias. ?format=types overlays same-activity catalog ids (falling back to the full list when activity is still null). Follow-up on the same PR: OpenRouter activity from architecture.output_modalities (video → synthesised videos/{id}; image-only has no classified route); BFL v1/{rawId}; BytePlus chat/image/video plus Seed Speech TTS vs ASR.
• 18.3 Fill contextWindow / maxOutput / modalities / request-feature capabilities on grain=provider rows (issue #53; pricing is a separate PR). Native /models payloads publish none of these (OpenAI, xAI) or only limits plus a capability tree (Anthropic Models API, since 2026-03) or limits + thinking/sampling defaults (Gemini). The rest comes from each provider's own published sources — never a third-party catalog (a model catalog that mirrors another caps its accuracy at theirs): OpenAI per-model markdown pages (developers.openai.com/api/docs/models/{slug}.md, fetched only for the ~65 pages the listed ids resolve to, dated snapshots via their alias), xAI's /v1/{language,image-generation,video-generation}-models for modalities plus docs.x.ai/docs/models.md for context windows. Adapters: openAiCompatModelFacts maps the extension fields OpenAI-compatible hosts already put on their rows (context_window / context_length / max_context_length, max_completion_tokens, input_modalities, supports_* booleans, groq/jina supported_features + supported_sampling_parameters, mistral capabilities) — one mapper in listOpenAiCompatibleModels covers groq, fireworks, moonshot, sambanova, hyperbolic, novita (context_size, hyphenated features); mistral, jina and cohere (features) call it too. Speech/media hosts (elevenlabs, deepgram, cartesia, stability, replicate, bfl, fal, byteplus, reactor) keep their own capability objects — token limits and request params don't apply; deepseek, cerebras, dashscope, perplexity, together publish bare rows. Output uses OpenRouter's shapes: file for documents, supported_parameters names as feature flags (tools, tool_choice, reasoning, reasoning_effort, temperature, top_p, top_k, structured_outputs, response_format). Nothing is inferred: a value is stored only when a provider field or table cell states it. So Grok has no capabilities (xAI publishes no feature flags), Gemini has no modalities (not on the row), a lone image/vision boolean does not produce a modalities object, tool_choice needs its own flag, and Anthropic's tool use, sampling params and Fable-tier mandatory thinking stay unset (prose only). Those prose-only facts are the target of the AI docs-extraction follow-up, which will attach the backing quote to each value. Gemini capabilities no longer holds supportedGenerationMethods; the :predict vs :generateContent decision those methods drive is now made at list time and stored as models.schema_endpoint_id (migration 0005), which the read path prefers over the config binding. The config fallback still reads leftover supportedGenerationMethods from capabilities so un-repolled Imagen rows keep :predict — first-party data kept, no id-prefix guessing. Deploy needs db:migrate:remote first (deploys do not run migrations). Docs parses are memoised in-isolate for 6h and fail closed: zero parsed rows throws and fails that provider's poll for the tick rather than nulling populated rows (which would fan out a bogus model.updated per model and again on recovery). Gotcha: the first poll after deploy legitimately emits one model.updated per newly filled row.
• 18.4 Bound-schema catalog facts + per-field provenance (issue #53). After listing/docs fill, the poller walks the bound generation request schema (schemaEndpointId → current input schema_versions) for OpenRouter supported_parameters names and input modalities. Listing/docs win on a field they stated; the schema fills remaining flags. generated specs are skipped (OpenAI-borrowed documents must not stamp tools onto DeepSeek/DashScope). Host-native capability objects (FAL/Reactor/…) are left alone. Each stored value carries models.fact_sources (migration 0006): per-field and per-flag { derivation, sourceUrl, endpointId, path }. GET /v1/models/{p}/{id} always returns factSources plus discrepancies against the already-polled OpenRouter catalog (join author/rawId; no extra fetch, no unlabeled OpenRouter copy). GET /v1/models?provenance=1 includes factSources on list rows. OpenRouter fill-nulls, pricing, and docs-extracted remain later slices.