Skip to content

feat(indexer): add Prometheus /metrics endpoint with indexer and transfer counters - #175

Open
royalTreasure wants to merge 1 commit into
Miracle656:mainfrom
royalTreasure:feat/prometheus-metrics
Open

feat(indexer): add Prometheus /metrics endpoint with indexer and transfer counters#175
royalTreasure wants to merge 1 commit into
Miracle656:mainfrom
royalTreasure:feat/prometheus-metrics

Conversation

@royalTreasure

Copy link
Copy Markdown

Closes #39

Summary

Wraith runs as a persistent background service with no operational metrics — a stalled indexer or a degrading RPC endpoint was visible only in the logs. This adds a Prometheus scrape endpoint and instruments the indexer, the RPC retry path, and the hot database operations.

Metrics

New src/metrics.ts, using prom-client:

Metric Type Labels
ledgers_indexed_total counter network
transfers_stored_total counter network, type (fungible / nft)
rpc_errors_total counter outcome (retry / exhausted)
last_indexed_ledger gauge network
db_query_duration_seconds histogram operation

Standard process_* / nodejs_* collectors are registered alongside them.

Two details worth calling out:

  • The ledger counter takes the per-poll delta, not the absolute sequence. A process resuming from its DB cursor would otherwise report several million ledgers indexed in one second on every restart.
  • rpc_errors_total counts attempts, not calls. withRetry hides transient failures from its callers by design, so counting only calls that exhausted their retries would read zero right up until the indexer falls over. outcome keeps the two readings separable.

Everything registers into a module-local Registry rather than prom-client's global default — the global is process-wide state shared with any dependency that also uses prom-client, and it cannot be cleared between tests without clobbering theirs.

Endpoint

GET /metrics returns the exposition format with prom-client's content type. It reads in-process counters only — no DB, no RPC — so it keeps answering while the subsystems it reports on are down, which is the point of having it. For the same reason it is exempt from:

  • the API rate limiter (a scrape endpoint that starts 429ing goes blind exactly when load is high enough to matter), and
  • the stale-read middleware's RPC health probe (which would add a network round-trip to every scrape and report nothing useful).

Registered in the OpenAPI document (src/openapi/build.ts, openapi.json regenerated via npm run docs:openapi).

Instrumentation

  • src/indexer.ts — ledger progress and stored-row counts on both the single-poll and the parallel (INGEST_WORKERS > 1) paths, including the empty-batch case where the cursor still advances.
  • src/rpc.tswithRetry records each failed attempt.
  • src/db.tsobserveDbQuery wraps upsertTransfers, upsertNftTransfers, getLastIndexedLedger, setLastIndexedLedger, and queryTransfers. Failures are timed too: a query that runs for eight seconds and then throws is exactly the one worth seeing on a latency graph.

/status

Now reports last_indexed_ledger alongside the existing lastIndexedLedger, matching the gauge's name. It is an additive alias — the camelCase field is untouched and both always carry the same value.

Tests

src/__tests__/metrics.test.ts (8 tests): valid exposition format and content type, all five custom metrics present with the correct # TYPE, recorded samples rendered with their labels, the endpoint still serving 200 while DB and RPC both reject, observeDbQuery timing both the success and the throwing path (and rethrowing), and /status carrying both ledger field names.

Verification

  • npm test — 298 passed across 27 suites (was 290/26); coverage thresholds still met.
  • npx tsc --noEmit — clean.
  • npm run docs:openapi — regenerated, diff is the /metrics path only.

…sfer counters

Wraith runs as a persistent background service with no operational metrics: a
stalled indexer or a degrading RPC endpoint was visible only in the logs.

New src/metrics.ts registers five custom metrics into a module-local registry
(not prom-client's process-global default, which is shared state no test can
clear safely) alongside the standard process/Node collectors:

  ledgers_indexed_total      counter   {network}
  transfers_stored_total     counter   {network, type}
  rpc_errors_total           counter   {outcome}
  last_indexed_ledger        gauge     {network}
  db_query_duration_seconds  histogram {operation}

The ledger counter takes the per-poll delta rather than the absolute sequence,
so a process resuming from a DB cursor does not report millions of ledgers
indexed in one second on every restart. rpc_errors_total counts attempts, not
calls: withRetry hides transient failures by design, so per-call counting would
read zero right up until the indexer falls over.

GET /metrics reads in-process counters only — no DB, no RPC — so it keeps
answering while the subsystems it reports on are down, and it is exempt from
both the rate limiter and the stale-read RPC probe for the same reason.

/status gains last_indexed_ledger as a snake_case alias of lastIndexedLedger,
matching the gauge name; the existing field is untouched.
@drips-wave

drips-wave Bot commented Aug 30, 2026

Copy link
Copy Markdown

@royalTreasure Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Prometheus /metrics endpoint

1 participant