Skip to content

feat(traffic): read endpoints + CLI status/sources/events (Phase 2 slice 2)#431

Merged
arberx merged 5 commits intomainfrom
arberx/post-pr429-next-stage
May 9, 2026
Merged

feat(traffic): read endpoints + CLI status/sources/events (Phase 2 slice 2)#431
arberx merged 5 commits intomainfrom
arberx/post-pr429-next-stage

Conversation

@arberx
Copy link
Copy Markdown
Member

@arberx arberx commented May 8, 2026

Summary

Adds the read surface that slice 1 deferred so an agent can see what syncs ingested without a DB query: three GET routes (/traffic/sources, /traffic/sources/:id with last-24h totals + latest run, /traffic/events with kind/since/until/limit/sourceId filters), three matching CLI commands (traffic status, traffic sources, traffic events) all with --format json, plus ApiClient methods and OpenAPI entries. Folds in the two correctness fixes flagged in the slice 1 review: upstream and auth-token failures now throw providerError() (502) so CLI exits 2 (system error, retryable) rather than 1 (user error), and the failed-run + source updates share a transaction so a mid-handler crash can't leave the run failed but the source still "connected". Events totals reflect the full window even when limit truncates the rows array, so a paginated query never undercounts. MCP toolkit and UI remain deferred to slice 3+.

Test plan

  • pnpm test — 2138 tests pass workspace-wide (+18 new: 13 route tests, 5 CLI tests)
  • pnpm typecheck clean
  • pnpm lint clean (pre-commit hook ran)
  • Smoke canonry traffic events <project> --kind crawler --format json against a synced project

@arberx arberx force-pushed the arberx/post-pr429-next-stage branch 3 times, most recently from a954e97 to 3b68b42 Compare May 8, 2026 15:33
arberx and others added 5 commits May 9, 2026 12:46
…ice 2)

Adds the read surface that slice 1 deferred so an agent can see what
syncs ingested without a DB query. Three new GET routes (`/traffic/sources`,
`/traffic/sources/:id` with last-24h totals + latest run, `/traffic/events`
with kind/since/until/limit/sourceId filters), three matching CLI commands
(`traffic status`, `traffic sources`, `traffic events`) all with --format
json, and ApiClient methods + OpenAPI entries.

Folds in the two correctness fixes flagged in the slice 1 review:
upstream and auth-token failures now throw providerError() (502) so CLI
exits 2 (system error, retryable) rather than 1 (user error), and the
failed-run + source updates share a transaction so a mid-handler crash
can't leave the run failed but the source still "connected".

Events totals reflect the full window even when limit truncates the rows
array, so a paginated query never undercounts. MCP toolkit and UI remain
deferred to slice 3+.
…es the CLI

Non-numeric values (e.g. `--since-minutes abc`) flowed through `parseInt` as
NaN and crashed the CLI with `RangeError: Invalid time value` at
`new Date(NaN).toISOString()`. `--limit abc` round-tripped to the API as
`limit=NaN` and bounced back as a 400 instead of being caught locally.
Switch both options to the existing `parseIntegerOption` helper so bad input
fails fast with a usage error.

Co-Authored-By: Claude Opus 4.7 <[email protected]>
Adds the MCP toolkit and the dashboard surface for the server-side traffic
ingestion shipped in slices 1+2, closing API → CLI → MCP → UI parity.

Slice 3 — MCP toolkit
- Promote 5 traffic ops to `included` in openapi-classification.ts
- Add 5 tools to tool-registry.ts (sources_list, source_get, events,
  connect_cloud_run, sync), tagged with the new `traffic` tier
- Add `traffic` toolkit to toolkits.ts so it loads on demand alongside
  monitoring/setup/gsc/ga/agent
- Wire input schemas to packages/contracts Zod schemas
- Tests cover registry tier counts, eager-mode count, handler dispatch,
  plus a stdio smoke test that loads the toolkit and calls one read
  (sources_list) + one write (sync)
- Aero auto-picks them up via mcp-to-agent-tool.ts (no AERO_EXCLUDED entry)

Slice 4 — UI dashboard
- New /traffic top-level page: project picker + sources table with
  status, last sync, 24h crawler / AI-referral hits, sample count, plus
  a Connect-Cloud-Run drawer (paste-or-upload SA JSON, write-only key)
- New /traffic/$projectName/$sourceId detail page: ScoreGauge tiles,
  ToneBadge status, latest run summary, hourly stacked-bar chart via
  ChartPrimitives (Recharts only — no direct import), filterable events
  table by kind / window
- React Query hooks in queries/server-traffic.ts; toneFromTrafficSourceStatus
  helper centralizes the status → MetricTone map
- Sidebar entry (Radar icon), mobile nav entry, breadcrumb mapping
- traffic-sync run completion now invalidates the new server-traffic
  query namespace alongside the existing GA traffic keys
- UI/CLI parity: every metric on screen comes from the same DTOs returned
  by `canonry traffic status/sources/events --format json`

Bumps to 4.15.0.

Co-authored-by: Claude Opus 4.7 <[email protected]>
…point

Three corrections from review feedback against the server-side ingestion plan:

1. `latestRun` on `GET /traffic/sources/:id` was filtered only by project +
   `traffic-sync`, so any second source seeded for the same project would
   surface another source's run. Add nullable `runs.source_id` (migration 51),
   populate it on traffic-sync insert, and require `runs.sourceId === row.id`
   in the detail query. Test exercises the multi-source isolation path
   directly.

2. `canonry traffic status --format json` was a client-side fan-out
   (list-sources + N×detail). That violates the single-call read rule and
   leaves the dashboard re-implementing the same fan-out. Add
   `GET /traffic/status` returning `{ sources: TrafficSourceDetailDto[] }`,
   wire the CLI through `ApiClient.trafficStatus`, and assert the new
   endpoint's per-source entries match `/traffic/sources/:id` byte-for-byte.

3. `latestRun.status` in the contract was `z.string()` instead of the
   canonical run-status enum. Switched to `runStatusSchema` so MCP / agent
   clients get the same machine contract as other run DTOs.

Also surfaces `traffic` in `canonry --help` (under Integrations) and adds it
to the telemetry `SUBCOMMAND_COMMANDS` set so subcommands track properly.

Cursor / `insertId` dedupe in sync and the canonical
`/traffic/{crawlers,referrals,timeline}` surfaces remain deferred to slice 3.

Co-Authored-By: Claude Opus 4.7 <[email protected]>
Slice 3 added the traffic toolkit; the new /traffic/status composite needs a
matching tool so agents can use the single-call read instead of N+1
list+detail. Mirrors the pattern of canonry_traffic_sources_list /
canonry_traffic_source_get and ships under the existing 'traffic' tier.

Test counts updated: total tools 72→73, read tools 48→49, traffic toolkit
5→6, eager stdio listing 74→75.

Co-Authored-By: Claude Opus 4.7 <[email protected]>
@arberx arberx force-pushed the arberx/post-pr429-next-stage branch from 3cd2d08 to 2e40a52 Compare May 9, 2026 16:46
@arberx arberx merged commit 1eb7d44 into main May 9, 2026
2 checks passed
@arberx arberx deleted the arberx/post-pr429-next-stage branch May 9, 2026 17:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant