Skip to content

[Backend] Webhook Retry Mechanism β€” Failed Deliveries Are Never RetriedΒ #353

@ogazboiz

Description

@ogazboiz

πŸ’¬ Want to contribute? Join us on Telegram: https://t.me/+DOylgFv1jyJlNzM0

Problem

backend/src/services/webhookService.ts sends webhooks with a single attempt (line 189: attempt_count, 1). If a delivery fails (timeout, 5xx response, network error), it is recorded in webhook_deliveries but never retried.

This means webhook subscribers can silently miss events with no way to recover other than re-polling.

Expected Behavior

Failed webhook deliveries should be retried with exponential backoff (e.g., 1m, 5m, 30m) up to a configurable max retry count.

Implementation Suggestion

  1. Add a max_attempts column to webhook_subscriptions (default 5)
  2. Add a scheduled job that queries webhook_deliveries for failed deliveries where attempt_count < max_attempts and delivered_at IS NULL
  3. Retry with exponential backoff
  4. Mark as permanently failed after max attempts

Files to Change

  • backend/src/services/webhookService.ts β€” add retry logic
  • backend/migrations/ β€” add migration for max_attempts column
  • New: backend/src/services/webhookRetryScheduler.ts

Acceptance Criteria

  • Failed deliveries are retried up to N times with exponential backoff
  • webhook_deliveries.attempt_count is incremented on each retry
  • Permanently failed deliveries are marked distinctly
  • Retry scheduler runs on a configurable interval
  • Admin endpoint to view retry queue status

Metadata

Metadata

Assignees

Labels

Stellar WaveIssues in the Stellar wave program

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions