Repository navigation
feat(dashboards): Omi TV device-vs-organic funnel row and gateway errors by lane - #24374
Merged
Merged
Conversation
…ors by lane Omi TV v3 gains a scroll-down funnel row under the fixed wall: the latest matured weekly new-user cohort split by "paired an Omi within 48 hours" (device) vs organic, step by step (share, onboarded, conversation day 1, conversation week 1, activated, kept weeks 2-5 from an older cohort), the weekly activation trend for six cohorts, and activation by entry platform. Install and first summary are not shown: PostHog sees nobody before sign-up, and first summary has no event. The funnel scan runs 28 days past each period's end so Kept (days 7-35) is complete for the older cohort; ending the scan at the period end cut that window to under a week. Core Features gains "Gateway errors by lane" and "Gateway error rate by lane", read from the lane_outcome:llm_gateway_requests:rate5m_offset5m recording rule. Thresholds are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
3 issues found across 7 files
Confidence score: 3/5
funnelFramescan hide available period-5 retention when the period-4 read fails, despite the panel promising the newest mature cohort. Select the lowest available period at or after period 4.wall-series.test.tshas nounknownrows, so the expected handling in thefunnelandOrganicsplit is unverified. Add anunknownrow and assert the expected behavior.- The platform-order test in
wall-series.test.tsuses equal iOS and Android counts, so it passes even if sorting is removed. Give Android a larger count so the assertion checks the order.
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="web/admin/lib/__tests__/wall-series.test.ts">
<violation number="1" location="web/admin/lib/__tests__/wall-series.test.ts:1731">
P2: The fixture has no `unknown` entry rows, so the stated behavior is untested. Add an `unknown` row and assert that it counts in the organic share of `funnel` and `Organic` of the split, but does not appear in `funnelPlatforms`.</violation>
<violation number="2" location="web/admin/lib/__tests__/wall-series.test.ts:1761">
P2: The "orders platforms by size" test cannot detect a missing sort. iOS and Android both have 100 people, so the expected order matches insertion order. Give Android a larger count in this fixture so the assertion depends on the size sort.</violation>
</file>
<file name="web/admin/lib/wall-series.ts">
<violation number="1" location="web/admin/lib/wall-series.ts:668">
P2: `funnelFrames` hard-codes Kept to period 4, so a transient period-4 read failure hides available period-5 retention even though the panel promises the newest mature cohort. Select the lowest available period at or after `FUNNEL_KEPT_PERIOD`.</violation>
</file>
Reply with feedback, questions, or to request a fix.
View guided diff | Turn on auto-fix | Re-trigger cubic
| describe("funnelFrames", () => { | ||
| const now = new Date("2026-11-02T12:00:00Z"); | ||
| // period, entry, device, people, onboarded, conv1, conv7, activated, kept | ||
| const deviceFunnel = Object.assign( |
There was a problem hiding this comment.
P2: The fixture has no unknown entry rows, so the stated behavior is untested. Add an unknown row and assert that it counts in the organic share of funnel and Organic of the split, but does not appear in funnelPlatforms.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At web/admin/lib/__tests__/wall-series.test.ts, line 1731:
<comment>The fixture has no `unknown` entry rows, so the stated behavior is untested. Add an `unknown` row and assert that it counts in the organic share of `funnel` and `Organic` of the split, but does not appear in `funnelPlatforms`.</comment>
<file context>
@@ -1723,3 +1724,61 @@ describe("variable margin frames (David 2026-10-23)", () => {
+describe("funnelFrames", () => {
+ const now = new Date("2026-11-02T12:00:00Z");
+ // period, entry, device, people, onboarded, conv1, conv7, activated, kept
+ const deviceFunnel = Object.assign(
+ [
+ [0, "ios", 1, 40, 36, 20, 30, 28, 0],
</file context>
| }); | ||
| }); | ||
|
|
||
| it("orders platforms by size and leaves an absent split empty", () => { |
There was a problem hiding this comment.
P2: The "orders platforms by size" test cannot detect a missing sort. iOS and Android both have 100 people, so the expected order matches insertion order. Give Android a larger count in this fixture so the assertion depends on the size sort.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At web/admin/lib/__tests__/wall-series.test.ts, line 1761:
<comment>The "orders platforms by size" test cannot detect a missing sort. iOS and Android both have 100 people, so the expected order matches insertion order. Give Android a larger count in this fixture so the assertion depends on the size sort.</comment>
<file context>
@@ -1723,3 +1724,61 @@ describe("variable margin frames (David 2026-10-23)", () => {
+ });
+ });
+
+ it("orders platforms by size and leaves an absent split empty", () => {
+ const { funnelPlatforms } = funnelFrames({ deviceFunnel }, now);
+ expect(funnelPlatforms).toEqual([
</file context>
| ...FUNNEL_STEPS.map(([step, column]) => split(step, latest, column)), | ||
| ] | ||
| : []; | ||
| const older = week(FUNNEL_KEPT_PERIOD); |
There was a problem hiding this comment.
P2: funnelFrames hard-codes Kept to period 4, so a transient period-4 read failure hides available period-5 retention even though the panel promises the newest mature cohort. Select the lowest available period at or after FUNNEL_KEPT_PERIOD.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At web/admin/lib/wall-series.ts, line 668:
<comment>`funnelFrames` hard-codes Kept to period 4, so a transient period-4 read failure hides available period-5 retention even though the panel promises the newest mature cohort. Select the lowest available period at or after `FUNNEL_KEPT_PERIOD`.</comment>
<file context>
@@ -601,6 +615,89 @@ const CAPTURE_LABELS: Record<(typeof CAPTURE_SEGMENTS)[number], string> = {
+ ...FUNNEL_STEPS.map(([step, column]) => split(step, latest, column)),
+ ]
+ : [];
+ const older = week(FUNNEL_KEPT_PERIOD);
+ if (people && count(older, PEOPLE))
+ funnel.push(split("Kept, weeks 2–5 (older cohort)", older, KEPT));
</file context>
Suggested change
| const older = week(FUNNEL_KEPT_PERIOD); | |
| const olderPeriod = all | |
| .map((r) => Number(r[0])) | |
| .filter((p) => p >= FUNNEL_KEPT_PERIOD) | |
| .sort((a, b) => a - b)[0]; | |
| const older = Number.isFinite(olderPeriod) ? week(olderPeriod) : []; |
…ing clients Review round 1 (Luna): - Device Paired is also sent for Apple Watch, Limitless and other vendors. "Device" now requires device_vendor 'omi' (mobile and macOS both send it) or type omi/openglass for older mobile builds. Live: 48 of 379 "device" users in the latest cohort had paired only non-Omi hardware. - Windows and web never send Onboarding Completed, so the Onboarded step now counts only people who arrived on iOS, Android or macOS, and says so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Omi TV v3: a funnel row below the fixed wall (scroll down; the 27-row TV wall itself is unchanged). The row splits new users into "device" (paired Omi hardware within 48 hours of first seen;
Device Pairedwithdevice_vendoromi, ortypeomi/openglass on older mobile builds) and organic. Pairing only another vendor (Apple Watch, Limitless, ...) counts as organic. David ruled 11-02 that the funnel shows as tiles only, with "paired" standing in for device owners.Onboarding Completed), conversation on day 1, conversation in week 1, activated (2+ active days in week 1). Kept (activated, then active in days 7–35) comes from the cohort four weeks older, the newest one old enough to measure it.Core Features: gateway errors by lane (David ruled "add it" on 11-02). These are two panels in the AI responses row:
Both read the
lane_outcome:llm_gateway_requests:rate5m_offset5mrecording rule that is already on main (#24167/#24184). Thresholds stay unchanged (green 99%+, red under 95%), and Core Features stays a separate dashboard from Omi TV.Data source and cost
The funnel is one new PostHog leg,
deviceFunnel, inside the existingactivationseries. It runs one HogQL read per weekly cohort, six in total, through the existingperiods()partial-retry and backoff path. A cohort-filtered read took about 3 s live.The scan runs 28 days past each period's end, so the days 7–35 Kept window is complete for the older cohort. Stopping the scan at the period end would have cut that window to under a week.
Live check (read-only, PostHog project 302298, 2026-11-02)
The organic denominator includes people with no client platform (
unknown: 346 in the latest cohort, with no onboarding or activity) and web sign-ups. This matches the wall's existing activation denominator (the same cohort ascaptureSourceQuery). The platform chart leavesunknownout.Activated can read slightly above "conversation, week 1", because activation counts foreground days as well as captures.
Tests
vitest run lib/__tests__/wall-series.test.ts: 61 passed, including the 5 newfunnelFramestests.python3 -m unittest discover web/admin/grafana: 16 passed. These cover the builder-matches-artifact check and the new funnel-row geometry check.backend/test.shwith the Core Features contract set (test_core_feature_headline_contract,test_daily_summary_monitoring_contract,test_monitoring_dashboard_query_budget,test_monitoring_notification_contract): 29 passed.make preflight: 20 checks passed.Deploy
omi-tv-v2.json.grafana_dashboard_provisioning.yml) foromi-core-features.json.No backend service deploy is needed.
Product invariants affected
none
🤖 Generated with Claude Code