feat(accounts): compute the per-day activity series from the lakehouse - #9317
Merged
Conversation
/api/v1/accounts/{ss58}/history returned day_count 0 for every account,
including hotkeys whose own /events feed is busy. All three surfaces fell
through to buildAccountHistory([], ...) -- the Postgres tier that owned
account_events_daily is gone and nothing replaced it.
MCP and GraphQL were worse than empty. Both call a shared loadAccountHistory
helper that ignored netuid, from, to and cursor entirely and returned the empty
series unconditionally, so a filtered request silently answered a different
question than the one asked. The helper now takes env, forwards all four, and
actually reads.
COMPUTED, NOT READ FROM THE ROLLUP. chain.account_events_daily DOES exist in
the lakehouse with the exact published shape, so porting the deleted query
verbatim would have been a much smaller diff. It is also a frozen export
spanning 2026-06-22 to 2026-07-15, so the route would answer "this account did
nothing since July 15" with nothing in the payload distinguishing that from a
measured zero -- the same decline-as-fact failure as #9303 and #9305.
chain.account_events is current to the head and the nightly rollup is a plain
GROUP BY, so it runs at request time instead. A live answer beats a small diff.
Two engine constraints shape the rest:
string_agg(DISTINCT event_kind, ',') -- what the retired writer used -- is
REJECTED, with the same scan-budget error that kills count(DISTINCT):
40015: scan budget exceeded: scanning too much data for string_agg(DISTINCT)
with GROUP BY
so the kinds get their own GROUP BY (day, netuid, event_kind) read joined here.
That read is bounded to the PAGE's day span, never the account's whole history:
a busy validator has ~216,000 (day, netuid, kind) groups all-time and a few
hundred inside one page. An empty page issues no kinds read at all, because the
unbounded scan would be the entire history to annotate nothing.
There is no OFFSET, and the cursor's (day, netuid) halves sit on opposite sides
of the aggregation, so SQL bounds the cursor to whole days and the exact tuple
boundary is applied after the fact -- over-fetching one day's worth of subnets
to guarantee the boundary is inside the window.
The day is published as YYYY-MM-DD rather than the engine's
2026-08-03T00:00:00.000000000Z, because that is the form the cursor encodes and
?from / ?to compare against. Hotkey-attributed, matching the retired rollup and
the published contract: /events matches hotkey OR coldkey and this route
documents that it does not.
Seven mutations are each caught: using string_agg, publishing the raw
timestamp, dropping the kinds read's day bound, making ?to exclusive, matching
coldkey too, emitting next_cursor on a short page, and skipping the cursor's
tuple refinement.
Closes #9315
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
metagraphed-data-api | e08b45e | Aug 03 2026, 09:05 PM |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
metagraphed-registry-sync-api | e08b45e | Aug 03 2026, 09:06 PM |
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
19 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
/api/v1/accounts/{ss58}/historyreturnedday_count: 0for every account, including hotkeyswhose own
/eventsfeed is busy.MCP and GraphQL were worse than empty: both call a shared
loadAccountHistoryhelper that ignorednetuid,from,toandcursorentirely and returned the empty series unconditionally. Bothsurfaces pass all four, so a filtered request silently answered a different question than the one
asked. The helper now takes
env, forwards all four, and actually reads.Computed, not read from the rollup
chain.account_events_dailyis in the lakehouse — 115,568 rows, the exact published shape. Averbatim port of the deleted query would have been a far smaller diff.
It is also a frozen export spanning 2026-06-22 → 2026-07-15. Reading it would make the route
answer "this account did nothing since July 15" with nothing in the payload distinguishing that
from a measured zero — the same decline-as-fact failure as #9303 and #9305.
chain.account_eventsis current to the head and the nightly rollup is a plainGROUP BY, so itruns at request time instead. Verified live:
Two engine constraints shaped the rest
string_agg(DISTINCT event_kind, ',')— exactly what the retired writer used — is rejected,with the same scan-budget error that kills
count(DISTINCT):So the kinds get their own
GROUP BY (day, netuid, event_kind)read, joined here. That read isbounded to the page's day span, never the account's whole history — a busy validator has
~216,000
(day, netuid, kind)groups all-time and a few hundred inside one page. An empty pageissues no kinds read at all, since the unbounded scan would be the entire history to annotate
nothing.
There is no
OFFSET, and the cursor's(day, netuid)halves sit on opposite sides of theaggregation — so SQL bounds the cursor to whole days and the exact tuple boundary is applied after
the fact, over-fetching one day's worth of subnets so the boundary is always inside the window.
dayis published asYYYY-MM-DD, not the engine's2026-08-03T00:00:00.000000000Z, because thatis the form the cursor encodes and
?from/?tocompare against. Hotkey-attributed, matching theretired rollup and the published contract (
/eventsmatches hotkey OR coldkey; this route documentsthat it does not).
Verification
Both queries run live against the lakehouse. Seven mutations, each caught:
string_agg(the rejected form)day?toexclusivecoldkeytoonext_cursoron a short pagePatch coverage 65/65 = 100% with every branch taken, measured by intersecting the diff's line
ranges with v8's uncovered set. The last six branches were engine-data guards (a non-string day, a
well-formed impossible date like
2026-13-45, an unusable netuid in a cursor page); I covered themrather than deleting them, since unlike a literal I control these come from outside.
1,520 tests pass across the affected suites;
tsc --noEmit,validate:contract-drift, prettier andeslint clean.
npm run buildproduces no artifact diff.Closes #9315