Skip to content

EventIngestionService processes ingested events sequentially #238

Description

@meshackyaro

Description

EventIngestionService.ingestEvents() and ingestSingleLedger() (src/event-ingestion/event-ingestion.service.ts) both loop over fetched events with for (const event of events) { const result = await this.eventProcessorService.processEvent(event); ... }, processing them one at a time even though EventProcessorService.processEvent() (src/event-ingestion/event-processor.service.ts) operations on different escrow IDs are largely independent. A ledger range with many events (up to the MAX_LEDGER_RANGE = 100-ledger window, each potentially containing multiple events) processes fully serially every 5-second poll tick (POLL_INTERVAL_MS), which could cause ingestion to fall behind the chain during bursts of on-chain activity.

Component

Backend

Difficulty

🔴 Hard

Tasks

  • Process events with bounded concurrency where safe (e.g. group by affected escrowId/topic to preserve required per-escrow ordering, but process independent escrows in parallel)
  • Be careful to preserve any implicit ordering guarantees the current sequential loop relies on (e.g. escrow_created before escrow_funded for the same escrow) — document the concurrency-safety analysis in the PR
  • Add a test with events for multiple independent escrows and a slow mock, asserting parallel processing doesn't reorder same-escrow events

Acceptance Criteria

  • Event processing throughput improves for bursts containing many independent-escrow events
  • Per-escrow event ordering is preserved and covered by a test

Estimated Time

1-2 days

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions