feat(analytics): ship PostHog v2 event schema

[lefarcen]

Why

Use case — product team handed off a refreshed埋点文档 2.0 last week (CSV in Drive) covering the v0.8 redesigned home, the new sidebar layout, the artifact toolbar overhaul, and a handful of brand-new pages (Automations / Plugins / Design Systems / Integrations). The previous v1 catalogue was built around the v0.7 layout and was missing roughly two-thirds of the new surfaces; the product team also asked for run_finished result=failed to carry an error_code classifier so dashboards can bucket agent failures, plus a guaranteed-fire page_view on every home render.

Pain being addressed — without v2 we have no way to attribute funnel drop-off to the new home, the new project modal, the four new top-level pages, or to slice agent failures by error_code. We also can't keep adding a per-page event name for every new product surface; the v1 model didn't scale past ~13 events.

What users will see

Nothing visible. This is wire-format only — no UI changes, no new copy. The PostHog dashboard team will see the new v2 events flowing on the next deploy with event_schema_version: 2; existing v1 dashboards continue to work against historical data via the schema-version filter.

Surface area

• API / contract — packages/contracts/src/analytics collapses 13+ v1 event names down to 4 core events (`page_view` / `ui_click` / `surface_view` / `*_result`), renames `anonymous_id` → `device_id`, and bumps `EVENT_SCHEMA_VERSION` to 2. New event interfaces cover every page / area / element combo in the v2 CSV.
• Default behavior change — the daemon now reads the new `x-od-analytics-device-id` request header instead of `x-od-analytics-anonymous-id`. Same value (the installationId), different header name; old web builds talking to a new daemon stop carrying analytics context until the matching web build ships.
• UI / Keyboard shortcut / CLI / Extension point / i18n / New dependency

Highlights

• Schema collapse. `page_view{page_name}` / `ui_click{page_name,area,element}` / `surface_view{page_name,area}` / discrete `*_result` events. Each surface owns a typed `*ClickProps` interface so call sites stay safe; the central `UiClickProps` union picks them up for `track()`.
• device_id rename + configure-state globals. The wire field for the distinct id is now `device_id` (value still installationId). The configure-state triplet (`has_available_configure_cli` / `configure_type` / `configure_availability`) is promoted to a PostHog `register()` call so every event inherits it; no more boilerplate at each helper.
• run_finished error_code. `run_created` / `run_finished` are now wired through the canonical `server.ts` `/api/runs` handler (a previous duplicate in `chat-routes.ts` was shadowed by Express's earlier registration and never executed — surfacing a separate latent bug where `run.clientType` was also never set, see "Adjacent fix" below). `run_finished result=failed` now carries a discrete `error_code` classifier: `run.errorCode` (explicit emit) → `AGENT_SIGNAL_` → `AGENT_EXIT_` → `AGENT_TERMINATED_UNKNOWN` fallback. Token totals and duration unchanged.
• Coverage. P0/P1/P2 call sites wired across every page: home nav / toolbar / chat composer / recent projects, projects list, automations, plugins (installed / available / sources tabs), design systems (modal + share popover), integrations (mcp / connectors / skills / use_everywhere), file manager, artifact toolbar / header / present popover / tweaks variants / share & export, feedback panel (with the v1 `clear_feedback_rating` rating-value bug fixed), settings sidebar / language / appearance / notifications / pets / privacy / connectors.
• PostHog UA filter opt-out. `opt_out_useragent_filter: true` so embedded webviews, fingerprinted browsers, and the Playwright-based e2e runs ingest captures. The Privacy → "Share usage data" toggle remains the single consent gate.

Adjacent fix — `run.clientType` now reaches Langfuse

`chat-routes.ts` declared a full `app.post('/api/runs', …)` handler that Express never used (the canonical handler in `server.ts` was registered first). Among the unreachable code: `run.clientType` was being set from `x-od-client` / Electron UA. Result: `langfuse-bridge.ts:320` has been shipping `run.clientType: undefined` for traces. The clientType inference moved to the canonical handler in this PR (single-line behavior change).

Verification

• `pnpm guard` / `pnpm typecheck` clean for daemon / web / contracts.
• `pnpm --filter @open-design/web test` — 1645 / 1645 passing.
• End-to-end through Playwright + the real PostHog ingest project 420348:
  • 28 unique page+area+element combos hit on a single home-to-projects-to-settings walk; every event landed with `event_schema_version: 2` and the expected `device_id` / `page_name` / `area` / `element` props.
  • `page_view` fired on all 8 page names: home / projects / automations / design_systems / plugins / integrations / chat_panel / file_manager.
  • `surface_view` fired on help_resources_popover / new_project_modal / plugin_replacement_modal.
  • `settings_view` fired across 9 areas including configure_execution_mode / language / appearance / notifications / pets / privacy.
  • `project_create_result` fired with `result=success`.
  • `settings_cli_test_result` fired with `result=success` from a real codex CLI test.
  • `run_created` + `run_finished` fired through a codex CLI run; result=failed because codex was misconfigured locally, and the new `error_code` field surfaced `AGENT_EXIT_1` — confirming the fallback chain works end-to-end.

Adjacent issues (out of scope)

• The remaining `*_result` events (`artifact_export_result` / `feedback_submit_result` / `file_upload_result` / `plugin_replacement_result` / `settings_byok_test_result` / `settings_connector_auth_result`) are wired in code but their UI paths ("upload a real file", "export a real artifact", "OAuth a real connector") were not automated in this PR's Playwright run. Production traffic will exercise them.
• The duplicate `reconcileAssistantMessageOnRunEnd` helper (chat-routes.ts had a local copy shadowed by the canonical one in server.ts) has been removed as part of this change.

[mrcfps]

@lefarcen I'm holding off on generating review comments for #2285 because this pull request has merge conflicts right now.

Please resolve the conflicts with main and push the updated branch. Once that's done, request or wait for the review to run again and I'll take another look.

_{🔁 Powered by Looper · runner=reviewer · agent=opencode · An autonomous AI dev team for your GitHub repos.}

[mrcfps]

@lefarcen Thanks for pushing the v2 analytics migration through. I walked the changed ranges and found two follow-up fixes worth landing before the new dashboards depend on this payload shape.

_{🔁 Powered by Looper · runner=reviewer · agent=opencode · An autonomous AI dev team for your GitHub repos.}

[lefarcen]

Thanks @mrcfps — both addressed in a7cd7928:

1. byokProtocolToTracking('senseaudio') now returns 'senseaudio' instead of falling through to null, so the SettingsDialog BYOK guards (if (byokProviderId)) no longer suppress SenseAudio's provider_option / field_focus / test_result captures.

2. Daemon-side run_created / run_finished now derive the configure-state triplet at capture time from detectAgents() + reqBody.agentId:

   • has_available_configure_cli: any CLI on PATH installed
   • configure_type: local_cli when the run targets an installed CLI, otherwise unknown (daemon can't see BYOK keys at this layer)
   • configure_availability: available / unavailable / unknown based on the requested agent's install status

   The BYOK arm staying unknown from the daemon is a deliberate compromise — the keys live in web-client storage and are not on the daemon's filesystem. If you'd rather the daemon mirror an authoritative web-side value, I can wire the web client to attach configureGlobals to the /api/runs body and have the daemon transit them through, but that's a larger change.

[mrcfps]

@lefarcen Thanks for addressing the earlier follow-ups. I walked the refreshed changed ranges on a7cd7928 and found one more analytics follow-up worth landing so the v2 payload matches the schema comments across the browser-side events.

_{🔁 Powered by Looper · runner=reviewer · agent=opencode · An autonomous AI dev team for your GitHub repos.}

[lefarcen]

Thanks for the follow-up @mrcfps — addressed in e0cfc5f9:

• Added a pure deriveConfigureGlobals() helper in `packages/contracts/src/analytics/events.ts` so the web client and the daemon derive the triplet from the same source of truth (covers all five `configure_type` buckets and three `configure_availability` buckets).
• `apps/web/src/App.tsx` now drives the triplet from a useEffect watching mode / agentId / apiKey / apiProtocolConfigs / agents, and re-registers it via `analytics.setConfigureGlobals(...)` whenever any of those change. The next capture inherits the fresh values.
• Exposed `setConfigureGlobals` through the `useAnalytics()` context (plus the test-stub) so consumers stay behind the provider boundary.
• Migrated the daemon-side derive in `server.ts` to the same helper so daemon-authoritative run_created / run_finished payloads match the web-side ones. The daemon arm passes `byokConfigured: undefined` because BYOK keys live in web-client storage; it falls back to the installed-CLI signal.
• New regression test at `apps/web/tests/analytics-configure-globals.test.ts` pins the derive matrix across all branches and confirms the setter actually mutates the client-side store. Locks the wiring so a future refactor can't silently regress the setter back into a no-op.

Verification: `pnpm guard` clean; daemon / web typecheck clean; web tests 1703/1703 passing (up from 1696 — 7 new tests in the configure-globals suite).

[mrcfps]

@lefarcen Thanks for iterating on the analytics follow-ups. I walked the latest changed ranges on e0cfc5f9 and found two remaining telemetry mismatches worth tightening before the v2 dashboards start depending on these fields.

_{🔁 Powered by Looper · runner=reviewer · agent=opencode · An autonomous AI dev team for your GitHub repos.}

[lefarcen]

Thanks @mrcfps — both addressed in c670570f:

1. DesignsTab projects page_view — added a once-per-mount trackPageView({ page_name: 'projects' }) with the same ref-keyed pattern HomeView / PluginsView use. Opening /projects now produces the v2 projects page-view event even when the user does not click anything.

2. ChatComposer hard-coded source — dropped the source: 'recent_project' constant entirely. You are right that this layer can't distinguish New-project / template / Recent-projects launches given the current router shape; over-attributing every chat to recent_project would corrupt the funnel slice this schema was meant to unlock. The field stays optional on PageViewProps, so when the router grows a launch-source channel we can populate it correctly in a follow-up PR. Better-no-source-than-wrong-source for now.

Verification: web typecheck clean; web tests 1703/1703 passing.

[mrcfps]

@lefarcen Thanks for tightening the analytics follow-ups on this head. I walked the refreshed changed ranges and found three non-blocking telemetry mismatches that are still worth fixing before the v2 dashboards rely on these result events.

_{🔁 Powered by Looper · runner=reviewer · agent=opencode · An autonomous AI dev team for your GitHub repos.}

[lefarcen]

Thanks @mrcfps — all three addressed in d4fb5a00:

1. plugin_replacement_result async outcome (HomeView.tsx): changed PendingReplacement.confirm to return Promise<void>, made the wrapper return the underlying usePlugin(...) promise, and moved the analytics emit into an async IIFE in the click handler. The success/failure branches now fire off the real settle so any plugin-apply rejection lands in the failed bucket.

2. file_upload_result heterogeneous batches (FileWorkspace.tsx): the implementation now maps every file in the batch to a tracking type, collapses to 'other' when more than one distinct type is present, and falls back to the single type otherwise. Matches the comment above the block.

3. project_create_result click→result correlation (NewProjectPanel.tsx): trackNewProjectModalElementClick() now accepts an optional { requestId }, and the create-button click threads the same id that project_create_result already carries.

Verification: web typecheck clean; web tests 1703/1703 passing.

[mrcfps]

@lefarcen Thanks for tightening the analytics follow-ups on this head. I walked the refreshed changed ranges and found two telemetry gaps that are still worth landing before the v2 dashboards start depending on these payloads.

_{🔁 Powered by Looper · runner=reviewer · agent=opencode · An autonomous AI dev team for your GitHub repos.}

[lefarcen]

Thanks @mrcfps — both addressed in d8e79eb3:

1. Cold-start configure-state gating (apps/web/src/App.tsx): the useEffect now waits on `agentsLoading === false` before pushing the triplet, so the first home/projects/plugins page_view inherits the boot defaults (`unknown`/`unknown`) rather than stamping `unavailable` on installed-CLI machines. Added 3 regression tests in `tests/analytics-configure-globals.test.ts` covering the empty-agents-with-daemon-mode and undefined-agents cases.

2. Daemon stopped reading unsent run_context fields (apps/daemon/src/server.ts): removed the `projectKind` / `entryFrom` / `projectSource` / `targetPlatforms` / `companionSurfaces` / `fidelity` / `connectors` / `useSpeakerNotes` / `includeAnimations` / `referenceTemplate` / `aspect` reads. The chat contract and `providers/daemon.ts` only thread `designSystemId` (which I kept), so the dropped fields were stamping null/undefined. A follow-up can extend the create-run payload to carry the full context properly; until then the daemon emits exactly what it can observe.

Verification: web typecheck clean; daemon typecheck clean; web tests 1706/1706 passing (up from 1703 — 3 new cold-start tests).

[mrcfps]

@lefarcen Thanks for tightening the analytics follow-ups on this head. I walked the current changed ranges and found one remaining telemetry classification mismatch worth fixing before the v2 run lifecycle dashboards depend on these fields.

_{🔁 Powered by Looper · runner=reviewer · agent=opencode · An autonomous AI dev team for your GitHub repos.}

[lefarcen]

Thanks @mrcfps — addressed in ad180c7a:

Pinned `mode: 'daemon'` on the `deriveConfigureGlobals(...)` call in `apps/daemon/src/server.ts`. The daemon's `/api/runs` handler is always a daemon-mode capture (BYOK lives in the web client), so the explicit mode tells the helper to judge `configure_availability` from the requested agent's install status rather than the cohort-wide fallback. A run aimed at an uninstalled agent now reports `unavailable` even when peers are on PATH.

Added a regression case in `tests/analytics-configure-globals.test.ts`:
```
{ mode: 'daemon', agentId: 'codex', agents: [{claude,true},{codex,false}] }
→ { has_available_configure_cli: true, configure_type: 'local_cli',
configure_availability: 'unavailable' }
```

Verification: daemon typecheck clean; web tests 1707/1707 passing (up from 1706 — 1 new regression test).

[mrcfps]

@lefarcen Thanks for pushing the analytics v2 follow-ups through. I walked the latest changed ranges on ad180c7a and found two remaining telemetry correctness issues worth tightening before the new dashboards depend on these page-view and feedback-correlation signals.

_{🔁 Powered by Looper · runner=reviewer · agent=opencode · An autonomous AI dev team for your GitHub repos.}

[mrcfps]

@lefarcen Thanks for pushing the analytics v2 follow-ups through. I walked the latest changed ranges and found two merge-safe analytics correctness gaps worth tightening before dashboards depend on this schema.

_{🔁 Powered by Looper · runner=reviewer · agent=opencode · An autonomous AI dev team for your GitHub repos.}

[mrcfps]

@lefarcen I'm holding off on generating review comments for #2285 because this pull request has merge conflicts right now.

Please resolve the conflicts with main and push the updated branch. Once that's done, request or wait for the review to run again and I'll take another look.

_{🔁 Powered by Looper · runner=reviewer · agent=opencode · An autonomous AI dev team for your GitHub repos.}