Problem
Submitting a Soroban transaction and waiting for confirmation takes several
seconds. Doing that inside a request handler ties up a connection, times out
under load, and loses the work entirely if the process restarts mid-flight.
What to do
- A durable job queue — BullMQ on Redis, or a Postgres-backed queue if the
dependency footprint matters more than throughput.
- Jobs for: transaction submission and confirmation polling, export generation,
indexer catch-up, webhook fan-out.
- Retry with backoff, a bounded attempt count, and a dead-letter queue that a
human can inspect.
- Idempotent job handlers — a job may run twice after a crash, and a
transaction submitted twice is a real financial event, not a retry.
- Visibility: queue depth, job age, and failure rate as metrics.
Acceptance criteria
Notes
Test the double-run case explicitly. "It only runs once" is an assumption that
holds until the first unclean shutdown.
Problem
Submitting a Soroban transaction and waiting for confirmation takes several
seconds. Doing that inside a request handler ties up a connection, times out
under load, and loses the work entirely if the process restarts mid-flight.
What to do
dependency footprint matters more than throughput.
indexer catch-up, webhook fan-out.
human can inspect.
transaction submitted twice is a real financial event, not a retry.
Acceptance criteria
Notes
Test the double-run case explicitly. "It only runs once" is an assumption that
holds until the first unclean shutdown.