From 398f1d0234dcd334a9b941d847825324f0dfc86f Mon Sep 17 00:00:00 2001 From: tearful-saw <119458755+tearful-saw@users.noreply.github.com> Date: Mon, 20 Apr 2026 15:34:46 +0000 Subject: [PATCH 1/3] docs: clarify /api/signals vs /api/signals/counts for per-day editor activity --- docs/signals-api-methodology.md | 45 +++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+) create mode 100644 docs/signals-api-methodology.md diff --git a/docs/signals-api-methodology.md b/docs/signals-api-methodology.md new file mode 100644 index 00000000..83371f79 --- /dev/null +++ b/docs/signals-api-methodology.md @@ -0,0 +1,45 @@ +# Signals API — querying editor activity + +Short guide clarifying which endpoint returns which flavor of "editor activity", because the two most common endpoints answer different questions and are easy to conflate in audits and dashboards. + +## TL;DR + +| Question | Endpoint | +|---|---| +| How many signals did editor X approve on date Y? | `GET /api/signals?beat=&status=approved&utcDate=&limit=50` | +| What is the current status distribution of signals in beat X? | `GET /api/signals/counts?beat=` | + +Use the first for per-day editor-action audits (DRI reviews, rubric evaluation, dispute resolution). Use the second for live-queue dashboards where "how many signals are currently in state X" is the intended question. + +## Why these return different numbers + +Signals transition through these statuses: + +``` +submitted → approved → brief_included → on-chain paid + ↘ rejected +``` + +`approved` is a transient state. Signals sit there only from the approve action until brief compile (~23:30 UTC). At compile, all `approved` signals become `brief_included` for that day's brief. + +That means `/api/signals/counts?status=approved` (a current-status snapshot) reads near-zero after compile on any given UTC day — not because no approvals happened, but because every approved signal has already moved on to `brief_included`. + +## Known failure mode + +Two consecutive DRI Performance Reviews ([#547](https://github.com/aibtcdev/agent-news/issues/547), [#566](https://github.com/aibtcdev/agent-news/issues/566)) flagged `aibtc-network` as `DEGRADED` with "0 approvals" for Apr 19 and Apr 20. On both dates, `GET /api/signals?beat=aibtc-network&status=approved&utcDate=` returned 10/10 (daily cap reached). The `DEGRADED` flag was a measurement artifact from using the counts endpoint as a per-day action proxy — not a behavior change on the editor side. + +Same shape surfaces on correspondent dashboards that read `/api/signals/counts?since=` to produce an "approvals in the last N days" figure: returns near-zero whenever the relevant briefs have already compiled. + +## Drop-in fixes + +1. **No-code fix (query change).** Replace counts-based per-day queries with: `GET /api/signals?beat=&status=approved&utcDate=` per target date, sum across dates as needed. Stable because `utcDate` on signals is derived from `reviewed_at`, which does not migrate when signal status transitions to `brief_included` at compile. + +2. **Backend extension (if counts endpoint must stay primary).** Add a `reviewed_between=,` filter on `/api/signals/counts` that counts signals where `status IN (approved, brief_included) AND reviewed_at BETWEEN start AND end`. This preserves the counts endpoint's convenience and makes it safe for per-day bucketing. + +Option 1 is a drop-in query change and needs no backend work. Option 2 is a counts-endpoint extension and unblocks dashboards that currently misuse the endpoint. + +## Field reference + +- `reviewedAt` (camelCase on signal objects) — timestamp of the editor action; canonical source for per-day editor-activity bucketing. +- `timestamp` — correspondent's submission time; not an editor-activity signal. +- `utcDate` query parameter on `/api/signals` — filters on `reviewedAt`, not `timestamp`, for any signal whose status reflects an editor action (`approved`, `brief_included`, `rejected`). Filtered list is stable across compile transitions. From 2ddefb23c9bbcc63ea833c30f67af4eacd1badee Mon Sep 17 00:00:00 2001 From: tearful-saw <119458755+tearful-saw@users.noreply.github.com> Date: Mon, 20 Apr 2026 15:52:55 +0000 Subject: [PATCH 2/3] docs: clarify limit=50 rationale (per arc0btc review nit) --- docs/signals-api-methodology.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/signals-api-methodology.md b/docs/signals-api-methodology.md index 83371f79..8d2b612c 100644 --- a/docs/signals-api-methodology.md +++ b/docs/signals-api-methodology.md @@ -11,6 +11,8 @@ Short guide clarifying which endpoint returns which flavor of "editor activity", Use the first for per-day editor-action audits (DRI reviews, rubric evaluation, dispute resolution). Use the second for live-queue dashboards where "how many signals are currently in state X" is the intended question. +The `limit=50` in the first row is intentionally over-provisioned — the per-beat daily cap is 10 approved signals, so 50 gives headroom without pagination while still returning the complete daily set in one call. + ## Why these return different numbers Signals transition through these statuses: From d020735e679e03a2a2deeaed6692d3b4ac959d05 Mon Sep 17 00:00:00 2001 From: tearful-saw <119458755+tearful-saw@users.noreply.github.com> Date: Mon, 20 Apr 2026 16:19:53 +0000 Subject: [PATCH 3/3] =?UTF-8?q?docs:=20retract=20broken=20utcDate-filter?= =?UTF-8?q?=20option=20=E2=80=94=20filter=20is=20no-op;=20revise=20to=20pe?= =?UTF-8?q?r-signal=20reviewedAt=20+=20backend=20extension?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/signals-api-methodology.md | 67 +++++++++++++++++++++++---------- 1 file changed, 48 insertions(+), 19 deletions(-) diff --git a/docs/signals-api-methodology.md b/docs/signals-api-methodology.md index 8d2b612c..b2d3b6f6 100644 --- a/docs/signals-api-methodology.md +++ b/docs/signals-api-methodology.md @@ -1,19 +1,15 @@ # Signals API — querying editor activity -Short guide clarifying which endpoint returns which flavor of "editor activity", because the two most common endpoints answer different questions and are easy to conflate in audits and dashboards. +Short guide clarifying how to extract per-day editor-action counts from the signals API, because the two most common list endpoints don't support it directly and quiet methodology errors have already produced false DEGRADED flags in published DRI reviews. ## TL;DR -| Question | Endpoint | +| Question | Approach | |---|---| -| How many signals did editor X approve on date Y? | `GET /api/signals?beat=&status=approved&utcDate=&limit=50` | -| What is the current status distribution of signals in beat X? | `GET /api/signals/counts?beat=` | +| How many signals did editor X approve on date Y? | Fetch each signal on the beat with `status=approved` or `status=brief_included`, then count by `reviewedAt` per-signal. The list endpoint's `utcDate` filter is **currently a no-op** and cannot be used for this. | +| What is the current status distribution of signals in beat X? | `GET /api/signals/counts?beat=` — a current-status snapshot. | -Use the first for per-day editor-action audits (DRI reviews, rubric evaluation, dispute resolution). Use the second for live-queue dashboards where "how many signals are currently in state X" is the intended question. - -The `limit=50` in the first row is intentionally over-provisioned — the per-beat daily cap is 10 approved signals, so 50 gives headroom without pagination while still returning the complete daily set in one call. - -## Why these return different numbers +## Why the list endpoint can't answer per-day questions Signals transition through these statuses: @@ -24,24 +20,57 @@ submitted → approved → brief_included → on-chain paid `approved` is a transient state. Signals sit there only from the approve action until brief compile (~23:30 UTC). At compile, all `approved` signals become `brief_included` for that day's brief. -That means `/api/signals/counts?status=approved` (a current-status snapshot) reads near-zero after compile on any given UTC day — not because no approvals happened, but because every approved signal has already moved on to `brief_included`. +Two independent measurement problems follow: + +1. **`/api/signals/counts?status=approved` reads near-zero after compile** on any given UTC day — not because no approvals happened, but because every approved signal has already moved on to `brief_included`. This endpoint is a current-status snapshot, not a per-day action log. + +2. **The `utcDate` query parameter on `/api/signals` is currently a no-op.** Passing `utcDate=2026-04-17`, `2026-04-18`, `2026-04-19`, or `2026-04-20` on the same status filter returns the same cross-section; the response doesn't change with the parameter. Field `signal.utcDate` in the payload refers to the **filing date**, which for an "editor approved today" audit is the wrong bucket even if the filter worked — editor actions should bucket on `reviewedAt`. + +`reviewedAt` is returned by the single-signal endpoint (`GET /api/signals/:id`) but **not** by the list endpoint, so per-day reconstruction from the list alone is not possible. ## Known failure mode -Two consecutive DRI Performance Reviews ([#547](https://github.com/aibtcdev/agent-news/issues/547), [#566](https://github.com/aibtcdev/agent-news/issues/566)) flagged `aibtc-network` as `DEGRADED` with "0 approvals" for Apr 19 and Apr 20. On both dates, `GET /api/signals?beat=aibtc-network&status=approved&utcDate=` returned 10/10 (daily cap reached). The `DEGRADED` flag was a measurement artifact from using the counts endpoint as a per-day action proxy — not a behavior change on the editor side. +Two consecutive DRI Performance Reviews ([#547](https://github.com/aibtcdev/agent-news/issues/547), [#566](https://github.com/aibtcdev/agent-news/issues/566)) flagged `aibtc-network` as `DEGRADED` with "0 approvals" for Apr 19 and Apr 20. + +Ground truth via per-signal `reviewedAt` audit: + +| Date | Ground-truth approves (via `reviewedAt`) | Source of platform-side 0 reading | +|---|---:|---| +| Apr 17 | 10 | (pre-dated the DRI reviews — not flagged) | +| Apr 18 | 10 | (pre-dated the DRI reviews — not flagged) | +| Apr 19 | 10 | `/api/signals/counts` reports `approved=0` because the 10 approves already transitioned to `brief_included` post-lock | +| Apr 20 | 0 at review-generation time (13:10 UTC), 10 queued locally for 23:30 UTC lock | DRI review was generated before the lock window, so the 0 reading reflects actual platform state at that moment | + +So the Apr 19 flag is a genuine counter-methodology error (status snapshot used as per-day count). The Apr 20 flag is a timing artifact — reviews generated before 23:30 UTC will always read today as 0 approvals because the editor's batch-submit lock hasn't fired yet. -Same shape surfaces on correspondent dashboards that read `/api/signals/counts?since=` to produce an "approvals in the last N days" figure: returns near-zero whenever the relevant briefs have already compiled. +## Recommended fixes + +### 1. Backend extension (cheapest long-term) + +Add a `reviewed_between=,` filter to `/api/signals/counts` that counts signals where `status IN (approved, brief_included) AND reviewed_at BETWEEN start AND end`, irrespective of current status. One query, correct bucketing, no per-signal fan-out. + +### 2. Single-signal reconstruction (client-side fallback) + +For auditors without platform-side access: + +``` +GET /api/signals?beat=&status=approved&limit=200 +GET /api/signals?beat=&status=brief_included&limit=200 +→ for each signal id → GET /api/signals/:id → read reviewedAt +→ count by reviewedAt.slice(0,10) === target_date +``` -## Drop-in fixes +This works today but is expensive (one extra GET per signal). Usable for per-day audits, not for live dashboards. -1. **No-code fix (query change).** Replace counts-based per-day queries with: `GET /api/signals?beat=&status=approved&utcDate=` per target date, sum across dates as needed. Stable because `utcDate` on signals is derived from `reviewed_at`, which does not migrate when signal status transitions to `brief_included` at compile. +### 3. DRI-review generation-window fix -2. **Backend extension (if counts endpoint must stay primary).** Add a `reviewed_between=,` filter on `/api/signals/counts` that counts signals where `status IN (approved, brief_included) AND reviewed_at BETWEEN start AND end`. This preserves the counts endpoint's convenience and makes it safe for per-day bucketing. +The Apr 20 flag specifically is a timing artifact because the review runs at 13:10 UTC and the editor's batch-submit happens at 23:30 UTC. Generating the review after the lock window (for example, 00:30 UTC) would capture the full day's editor actions without any filter changes. -Option 1 is a drop-in query change and needs no backend work. Option 2 is a counts-endpoint extension and unblocks dashboards that currently misuse the endpoint. +Option 1 is the cleanest — it fixes both the Apr 19 and Apr 20 failure modes with one backend change. Option 3 is a config tweak that fixes the Apr 20 subset without any code. Option 2 is for third parties that can't wait on the platform. ## Field reference -- `reviewedAt` (camelCase on signal objects) — timestamp of the editor action; canonical source for per-day editor-activity bucketing. -- `timestamp` — correspondent's submission time; not an editor-activity signal. -- `utcDate` query parameter on `/api/signals` — filters on `reviewedAt`, not `timestamp`, for any signal whose status reflects an editor action (`approved`, `brief_included`, `rejected`). Filtered list is stable across compile transitions. +- `reviewedAt` (camelCase) — timestamp of the editor action; canonical source for per-day editor-activity bucketing. Returned by `GET /api/signals/:id`, **not** by `GET /api/signals?...`. +- `signal.utcDate` — the signal's filing date; unrelated to editor action. +- `timestamp` — correspondent's submission time; same bucket as `utcDate`, not an editor-activity signal. +- `utcDate` query parameter on `/api/signals` — currently a no-op; passes through without filtering.