feat(traffic): Cloudflare Worker connect + ingest API surface#636
Draft
arberx wants to merge 2 commits into
Draft
feat(traffic): Cloudflare Worker connect + ingest API surface#636arberx wants to merge 2 commits into
arberx wants to merge 2 commits into
Conversation
Wires the Cloudflare Worker traffic source end-to-end on the server side — the operator can now POST /traffic/connect/cloudflare to get a generated Worker script and the deployed Worker can POST /traffic/cloudflare/ingest to push events into the rollup pipeline. Connect endpoint: - Generates per-source bearer token + HMAC secret + Worker script - Stores bearer hash on traffic_sources.ingest_token_hash; cleartext secrets to ~/.canonry/config.yaml under cloudflareTraffic.connections - Idempotent — reconnect rotates secrets, reuses source row - Writes audit log entry traffic.cloudflare.connected - No upstream probe (push model — Worker is deployed by the operator) Ingest endpoint: - Opts out of the global cnry_* bearer check (custom per-source auth) - Verifies bearer hash + HMAC-SHA256 signature + ±300s timestamp window - Single 401 envelope on any auth failure (no leg-disambiguation) - Normalizes events, runs the shared classifier + rollup pipeline, updates last_worker_version - Returns accepted/dropped counts + per-bucket counters Plus: - ApiClient.trafficConnectCloudflare() - MCP tool canonry_traffic_connect_cloudflare (ingest is excluded — wrong auth shape for an MCP caller) - OpenAPI spec + regenerated SDK - 19 new tests in packages/api-routes/test/traffic-cloudflare.test.ts (connect + ingest happy paths, every auth-failure leg, payload validation, AI-referral / AI-user-fetch routing, sample row write) - 12 new tests for the cloudflare-traffic-config helpers Stacked on arberx/server-log-integrations (foundation PR #635). Next stack: CLI commands + doctor checks + dashboard. Full workspace test passes (3412/3412). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
observedAt was set inside buildEvent, which only ran from forward() after event.waitUntil fired — i.e. after the upstream response resolved or rejected. The field reflected response completion (or full upstream timeout on the error path), not request entry, and could push events into the wrong hourly rollup bucket on slow origins. Capture observedAt synchronously at the top of the fetch handler and thread it through forward() → buildEvent() so the field matches its name. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Wires the Cloudflare Worker traffic source end-to-end on the server side. Stacked on top of #635 — the foundation PR carrying the contracts, schema, and integration package.
After this PR:
POST /api/v1/projects/:name/traffic/connect/cloudflareand get back a generated Worker script + wrangler.tomlPOST /api/v1/projects/:name/traffic/cloudflare/ingestConnect endpoint
POST /api/v1/projects/:name/traffic/connect/cloudflaresha256(bearer)ontraffic_sources.ingest_token_hash; cleartext secrets in~/.canonry/config.yamlundercloudflareTraffic.connections.<sourceId>traffic.cloudflare.connectedIngest endpoint
POST /api/v1/projects/:name/traffic/cloudflare/ingestThe unique endpoint in canonry — opts out of the global
cnry_*bearer check viashouldSkipAuthand carries its own per-source auth:X-Canonry-Source-IdheaderverifyRequestSignaturefromintegration-cloudflare-worker— ±300s timestamp window + HMAC-SHA256 over\${timestamp}.\${body}normalizeCloudflareWorkerEvent, runsbuildTrafficProbeReport, upserts hourly rollups in a single transaction, updateslast_worker_versionResponse includes
acceptedEvents,droppedEvents,workerVersionAck, plus per-bucket row counts so the Worker (or a debug curl) can see what landed.What's plumbed
api-routes→integration-cloudflare-workerauth.ts—shouldSkipAuthnow skips/traffic/cloudflare/ingestURLstraffic.ts—CloudflareTrafficCredentialStoreinterface, both routes,CLOUDFLARE_WORKER_VERSIONconstantindex.ts— wirescloudflareTrafficCredentialStore+cloudflareTrafficIngestUrlthroughTrafficRoutesOptionscanonry/src/config.ts—CloudflareTrafficConnectionConfigEntrycanonry/src/cloudflare-traffic-config.ts— CRUD helpers (NEW)canonry/src/server.ts— constructs the store from config, derives ingest URL frompublicUrl/apiUrl+basePathcanonry/src/client.ts—ApiClient.trafficConnectCloudflare()canonry_traffic_connect_cloudflare(write, idempotent). Ingest is classifiedexcluded-protocol— wrong auth shape for an MCP callerTests (TDD)
packages/api-routes/test/traffic-cloudflare.test.ts:last_worker_versionupdated; missing/wrong bearer/expired ts/unknown source-id/tampered body all return 401; empty events array / wrong schemaVersion return 400; AI-referral → referral bucket; AI-user-fetch UA → ai-user-fetch bucket; raw_event_samples row writtenpackages/canonry/test/cloudflare-traffic-config.test.tsmcp-registry.test.ts(85 → 86, traffic toolkit 10 → 11) andmcp-stdio.test.ts(87 → 88)traffic_sourcescolumns in the foundation PRFull workspace test passes (3412/3412), workspace typecheck clean, workspace lint clean (only pre-existing apps/web warnings).
Out of scope (next stacks)
🤖 Generated with Claude Code