Skip to content

feat(models/up): back health check with Analytics Engine; remove admin Model Status tab - #7072

Merged
chrarnoldus merged 3 commits into
mainfrom
remove-model-status-alerting
Oct 2, 2026
Merged

chrarnoldus merged 3 commits into
mainfrom
remove-model-status-alerting

Conversation

@chrarnoldus

@chrarnoldus chrarnoldus commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • /api/models/up, the model traffic health check polled by BetterStack, now reads the o11y_api_metrics Analytics Engine dataset instead of microdollar_usage. The URL, key, at parameter, time windows, thresholds, status codes and JSON format are unchanged.
  • Removed HEALTH_CHECK_EXCLUSIONS, so every monitored model can now return a 503. The per-model monitored field stays in the response and is always true.
  • Removed the Model Status tab from /admin/alerting, its ModelStatusContent component, and the /admin/model-status redirect page.

Query mapping

Before (microdollar_usage) After (o11y_api_metrics)
requested_model blob2 (resolved model), matched against normalizeModelId(monitoredModel)
has_error = false blob4 = '0'
COUNT(*) SUM(_sample_interval)
COUNT(DISTINCT kilo_user_id) count(DISTINCT blob7)

Windows are unchanged: current 15m, previous 15–30m, baseline 30m–2h (divided by 6). On a query error or a 10s timeout the endpoint still reports healthy (fails open). It uses the existing R2_ACCOUNT_ID and CF_ANALYTICS_ENGINE_TOKEN web env vars, which the KiloClaw and Gastown admin analytics routes already use. The emitted metrics data is unchanged.

Known differences from the Postgres version

  • The resolved model includes kilo-auto traffic routed to that model, and :free variants are counted together with their paid model.
  • Unique-user counts are not corrected for sampling, so they can be lower under heavy sampling. This only makes alerts less likely.
  • The check depends on the metrics pipeline (/ingest/api-metrics). If ingestion stops, models look like their traffic dropped.
  • Right after deploy, Analytics Engine has no data from before Revert "chore(ai-gateway): remove Cloudflare analytics (#6084)" #7071, which re-enabled metrics emission. Until 2h of data exists, baselines are below the 20-user minimum and the endpoint reports healthy.

Verification

  • Ran the generated SQL against production Analytics Engine with a historical anchor (2026-08-04 12:00 UTC). It returns current/previous/baseline buckets per model as expected, with numbers returned as strings, which the response schema coerces.
  • The rest is left to CI.

@chrarnoldus chrarnoldus self-assigned this Oct 1, 2026
@kilo-code-bot

kilo-code-bot Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Files Reviewed (4 files)
  • apps/web/src/app/admin/alerting/ModelStatusContent.tsx (deleted)
  • apps/web/src/app/admin/alerting/page.tsx
  • apps/web/src/app/admin/model-status/page.tsx (deleted)
  • apps/web/src/app/api/models/up/route.ts
Previous Review Summaries (2 snapshots, latest commit eaf8926)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit eaf8926)

Status: No Issues Found | Recommendation: Merge

Files Reviewed (4 files)
  • apps/web/src/app/admin/alerting/ModelStatusContent.tsx (deleted)
  • apps/web/src/app/admin/alerting/page.tsx
  • apps/web/src/app/admin/model-status/page.tsx (deleted)
  • apps/web/src/app/api/models/up/route.ts

Previous review (commit 39e67e8)

Status: No Issues Found | Recommendation: Merge

Files Reviewed (5 files)
  • apps/web/src/app/admin/alerting/ModelStatusContent.tsx (deleted)
  • apps/web/src/app/admin/alerting/page.tsx
  • apps/web/src/app/admin/model-status/page.tsx (deleted)
  • apps/web/src/app/api/code-reviews/up/route.ts
  • apps/web/src/app/api/models/up/route.ts (deleted)

Reviewed by deepseek-v4.1-flash · Input: 0 · Output: 0 · Cached: 0

Review guidance: REVIEW.md from base branch main

@chrarnoldus chrarnoldus changed the title chore(admin): remove model status alerting tab and /api/models/up endpoint chore(admin): remove model status tab; back /api/models/up with Analytics Engine Oct 1, 2026
@chrarnoldus
chrarnoldus force-pushed the remove-model-status-alerting branch from eaf8926 to 63bd102 Compare October 2, 2026 07:25
@chrarnoldus chrarnoldus changed the title chore(admin): remove model status tab; back /api/models/up with Analytics Engine feat(models/up): back health check with Analytics Engine; remove admin Model Status tab Oct 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants