Skip to content

feat(accounts): compute the per-day activity series from the lakehouse - #9317

Merged
JSONbored merged 1 commit into
mainfrom
feat/account-history-cold-tier
Aug 3, 2026
Merged

feat(accounts): compute the per-day activity series from the lakehouse#9317
JSONbored merged 1 commit into
mainfrom
feat/account-history-cold-tier

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

/api/v1/accounts/{ss58}/history returned day_count: 0 for every account, including hotkeys
whose own /events feed is busy.

MCP and GraphQL were worse than empty: both call a shared loadAccountHistory helper that ignored
netuid, from, to and cursor entirely
and returned the empty series unconditionally. Both
surfaces pass all four, so a filtered request silently answered a different question than the one
asked. The helper now takes env, forwards all four, and actually reads.

Computed, not read from the rollup

chain.account_events_daily is in the lakehouse — 115,568 rows, the exact published shape. A
verbatim port of the deleted query would have been a far smaller diff.

It is also a frozen export spanning 2026-06-22 → 2026-07-15. Reading it would make the route
answer "this account did nothing since July 15" with nothing in the payload distinguishing that
from a measured zero — the same decline-as-fact failure as #9303 and #9305.

chain.account_events is current to the head and the nightly rollup is a plain GROUP BY, so it
runs at request time instead. Verified live:

day netuid event_count first_block last_block
2026-08-03 127 72 8,759,894 8,765,497

Two engine constraints shaped the rest

string_agg(DISTINCT event_kind, ',') — exactly what the retired writer used — is rejected,
with the same scan-budget error that kills count(DISTINCT):

40015: scan budget exceeded: scanning too much data for string_agg(DISTINCT) with GROUP BY

So the kinds get their own GROUP BY (day, netuid, event_kind) read, joined here. That read is
bounded to the page's day span, never the account's whole history — a busy validator has
~216,000 (day, netuid, kind) groups all-time and a few hundred inside one page. An empty page
issues no kinds read at all, since the unbounded scan would be the entire history to annotate
nothing.

There is no OFFSET, and the cursor's (day, netuid) halves sit on opposite sides of the
aggregation — so SQL bounds the cursor to whole days and the exact tuple boundary is applied after
the fact, over-fetching one day's worth of subnets so the boundary is always inside the window.

day is published as YYYY-MM-DD, not the engine's 2026-08-03T00:00:00.000000000Z, because that
is the form the cursor encodes and ?from/?to compare against. Hotkey-attributed, matching the
retired rollup and the published contract (/events matches hotkey OR coldkey; this route documents
that it does not).

Verification

Both queries run live against the lakehouse. Seven mutations, each caught:

mutation tests failed
use string_agg (the rejected form) 1
publish the raw timestamp as day 8
drop the kinds read's day bound 1
make ?to exclusive 1
match coldkey too 1
emit next_cursor on a short page 1
skip the cursor's tuple refinement 1

Patch coverage 65/65 = 100% with every branch taken, measured by intersecting the diff's line
ranges with v8's uncovered set. The last six branches were engine-data guards (a non-string day, a
well-formed impossible date like 2026-13-45, an unusable netuid in a cursor page); I covered them
rather than deleting them, since unlike a literal I control these come from outside.

1,520 tests pass across the affected suites; tsc --noEmit, validate:contract-drift, prettier and
eslint clean. npm run build produces no artifact diff.

Closes #9315

/api/v1/accounts/{ss58}/history returned day_count 0 for every account,
including hotkeys whose own /events feed is busy. All three surfaces fell
through to buildAccountHistory([], ...) -- the Postgres tier that owned
account_events_daily is gone and nothing replaced it.

MCP and GraphQL were worse than empty. Both call a shared loadAccountHistory
helper that ignored netuid, from, to and cursor entirely and returned the empty
series unconditionally, so a filtered request silently answered a different
question than the one asked. The helper now takes env, forwards all four, and
actually reads.

COMPUTED, NOT READ FROM THE ROLLUP. chain.account_events_daily DOES exist in
the lakehouse with the exact published shape, so porting the deleted query
verbatim would have been a much smaller diff. It is also a frozen export
spanning 2026-06-22 to 2026-07-15, so the route would answer "this account did
nothing since July 15" with nothing in the payload distinguishing that from a
measured zero -- the same decline-as-fact failure as #9303 and #9305.
chain.account_events is current to the head and the nightly rollup is a plain
GROUP BY, so it runs at request time instead. A live answer beats a small diff.

Two engine constraints shape the rest:

string_agg(DISTINCT event_kind, ',') -- what the retired writer used -- is
REJECTED, with the same scan-budget error that kills count(DISTINCT):

  40015: scan budget exceeded: scanning too much data for string_agg(DISTINCT)
  with GROUP BY

so the kinds get their own GROUP BY (day, netuid, event_kind) read joined here.
That read is bounded to the PAGE's day span, never the account's whole history:
a busy validator has ~216,000 (day, netuid, kind) groups all-time and a few
hundred inside one page. An empty page issues no kinds read at all, because the
unbounded scan would be the entire history to annotate nothing.

There is no OFFSET, and the cursor's (day, netuid) halves sit on opposite sides
of the aggregation, so SQL bounds the cursor to whole days and the exact tuple
boundary is applied after the fact -- over-fetching one day's worth of subnets
to guarantee the boundary is inside the window.

The day is published as YYYY-MM-DD rather than the engine's
2026-08-03T00:00:00.000000000Z, because that is the form the cursor encodes and
?from / ?to compare against. Hotkey-attributed, matching the retired rollup and
the published contract: /events matches hotkey OR coldkey and this route
documents that it does not.

Seven mutations are each caught: using string_agg, publishing the raw
timestamp, dropping the kinds read's day bound, making ?to exclusive, matching
coldkey too, emitting next_cursor on a short page, and skipping the cursor's
tuple refinement.

Closes #9315
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 3, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
metagraphed-data-api e08b45e Aug 03 2026, 09:05 PM

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 3, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
metagraphed-registry-sync-api e08b45e Aug 03 2026, 09:06 PM

@superagent-security

Copy link
Copy Markdown

Superagent didn't find any vulnerabilities or security issues in this PR.

@JSONbored
JSONbored merged commit 8dd7100 into main Aug 3, 2026
7 checks passed
@JSONbored
JSONbored deleted the feat/account-history-cold-tier branch August 3, 2026 21:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

accounts/{ss58}/history returns zero days for every account, and MCP/GraphQL silently drop its filters

1 participant