feat(traffic): read endpoints + CLI status/sources/events (Phase 2 slice 2)#431
Open
feat(traffic): read endpoints + CLI status/sources/events (Phase 2 slice 2)#431
Conversation
arberx
force-pushed
the
arberx/post-pr429-next-stage
branch
2 times, most recently
from
6dd00ab to
20ec190
Compare
…ice 2) Adds the read surface that slice 1 deferred so an agent can see what syncs ingested without a DB query. Three new GET routes (`/traffic/sources`, `/traffic/sources/:id` with last-24h totals + latest run, `/traffic/events` with kind/since/until/limit/sourceId filters), three matching CLI commands (`traffic status`, `traffic sources`, `traffic events`) all with --format json, and ApiClient methods + OpenAPI entries. Folds in the two correctness fixes flagged in the slice 1 review: upstream and auth-token failures now throw providerError() (502) so CLI exits 2 (system error, retryable) rather than 1 (user error), and the failed-run + source updates share a transaction so a mid-handler crash can't leave the run failed but the source still "connected". Events totals reflect the full window even when limit truncates the rows array, so a paginated query never undercounts. MCP toolkit and UI remain deferred to slice 3+.
…es the CLI Non-numeric values (e.g. `--since-minutes abc`) flowed through `parseInt` as NaN and crashed the CLI with `RangeError: Invalid time value` at `new Date(NaN).toISOString()`. `--limit abc` round-tripped to the API as `limit=NaN` and bounced back as a 400 instead of being caught locally. Switch both options to the existing `parseIntegerOption` helper so bad input fails fast with a usage error. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
arberx
force-pushed
the
arberx/post-pr429-next-stage
branch
from
a954e97 to
3b68b42
Compare
13 tasks
Adds the MCP toolkit and the dashboard surface for the server-side traffic ingestion shipped in slices 1+2, closing API → CLI → MCP → UI parity. Slice 3 — MCP toolkit - Promote 5 traffic ops to `included` in openapi-classification.ts - Add 5 tools to tool-registry.ts (sources_list, source_get, events, connect_cloud_run, sync), tagged with the new `traffic` tier - Add `traffic` toolkit to toolkits.ts so it loads on demand alongside monitoring/setup/gsc/ga/agent - Wire input schemas to packages/contracts Zod schemas - Tests cover registry tier counts, eager-mode count, handler dispatch, plus a stdio smoke test that loads the toolkit and calls one read (sources_list) + one write (sync) - Aero auto-picks them up via mcp-to-agent-tool.ts (no AERO_EXCLUDED entry) Slice 4 — UI dashboard - New /traffic top-level page: project picker + sources table with status, last sync, 24h crawler / AI-referral hits, sample count, plus a Connect-Cloud-Run drawer (paste-or-upload SA JSON, write-only key) - New /traffic/$projectName/$sourceId detail page: ScoreGauge tiles, ToneBadge status, latest run summary, hourly stacked-bar chart via ChartPrimitives (Recharts only — no direct import), filterable events table by kind / window - React Query hooks in queries/server-traffic.ts; toneFromTrafficSourceStatus helper centralizes the status → MetricTone map - Sidebar entry (Radar icon), mobile nav entry, breadcrumb mapping - traffic-sync run completion now invalidates the new server-traffic query namespace alongside the existing GA traffic keys - UI/CLI parity: every metric on screen comes from the same DTOs returned by `canonry traffic status/sources/events --format json` Bumps to 4.15.0. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
…point
Three corrections from review feedback against the server-side ingestion plan:
1. `latestRun` on `GET /traffic/sources/:id` was filtered only by project +
`traffic-sync`, so any second source seeded for the same project would
surface another source's run. Add nullable `runs.source_id` (migration 51),
populate it on traffic-sync insert, and require `runs.sourceId === row.id`
in the detail query. Test exercises the multi-source isolation path
directly.
2. `canonry traffic status --format json` was a client-side fan-out
(list-sources + N×detail). That violates the single-call read rule and
leaves the dashboard re-implementing the same fan-out. Add
`GET /traffic/status` returning `{ sources: TrafficSourceDetailDto[] }`,
wire the CLI through `ApiClient.trafficStatus`, and assert the new
endpoint's per-source entries match `/traffic/sources/:id` byte-for-byte.
3. `latestRun.status` in the contract was `z.string()` instead of the
canonical run-status enum. Switched to `runStatusSchema` so MCP / agent
clients get the same machine contract as other run DTOs.
Also surfaces `traffic` in `canonry --help` (under Integrations) and adds it
to the telemetry `SUBCOMMAND_COMMANDS` set so subcommands track properly.
Cursor / `insertId` dedupe in sync and the canonical
`/traffic/{crawlers,referrals,timeline}` surfaces remain deferred to slice 3.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Slice 3 added the traffic toolkit; the new /traffic/status composite needs a matching tool so agents can use the single-call read instead of N+1 list+detail. Mirrors the pattern of canonry_traffic_sources_list / canonry_traffic_source_get and ships under the existing 'traffic' tier. Test counts updated: total tools 72→73, read tools 48→49, traffic toolkit 5→6, eager stdio listing 74→75. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the read surface that slice 1 deferred so an agent can see what syncs ingested without a DB query: three GET routes (
/traffic/sources,/traffic/sources/:idwith last-24h totals + latest run,/traffic/eventswithkind/since/until/limit/sourceIdfilters), three matching CLI commands (traffic status,traffic sources,traffic events) all with--format json, plus ApiClient methods and OpenAPI entries. Folds in the two correctness fixes flagged in the slice 1 review: upstream and auth-token failures now throwproviderError()(502) so CLI exits 2 (system error, retryable) rather than 1 (user error), and the failed-run + source updates share a transaction so a mid-handler crash can't leave the run failed but the source still "connected". Events totals reflect the full window even whenlimittruncates the rows array, so a paginated query never undercounts. MCP toolkit and UI remain deferred to slice 3+.Test plan
pnpm test— 2138 tests pass workspace-wide (+18 new: 13 route tests, 5 CLI tests)pnpm typecheckcleanpnpm lintclean (pre-commit hook ran)canonry traffic events <project> --kind crawler --format jsonagainst a synced project