retry with backoff and a circuit breaker for the price oracle calls

[mikewheeleer]

Summary

Calls to the price oracle fail transiently and, on sustained outages, pile up and exhaust resources. Add bounded retries with exponential backoff + jitter and a circuit breaker that fails fast when the price oracle is unhealthy.

Why this matters

Naive calls turn a transient blip into a cascading outage. Backoff + circuit breaking is the standard resilience pattern and protects the whole service.

Requirements

• Wrap the price oracle calls with retry (bounded attempts, exponential backoff + jitter).
• Add a circuit breaker: open after N consecutive failures, half-open probe after a cooldown, close on success.
• Only retry idempotent/safe operations; never retry a non-idempotent side effect blindly.
• Surface breaker state via a metric/log and a typed upstream_unavailable error when open.

Technical guidance

• Make attempts, backoff base, and thresholds configurable with safe defaults.
• Keep the breaker state per-dependency, not global.

Edge cases — each must have a test

• transient failure then success -> retried and succeeds
• N consecutive failures -> breaker opens, calls fail fast
• cooldown elapses -> half-open probe; success closes it
• non-retryable error -> not retried
• backoff respects the max attempt bound

Acceptance criteria

• All requirements and every edge case above implemented and covered by tests
• New unit and integration tests; existing tests still pass and no regressions
• Structured, typed errors (no leaked internals; stable codes)
• npm run lint, npm test, and npm run build all pass locally
• Code follows the repo's existing conventions; no duplication or dead code
• Short docs/comments explaining the design and any non-obvious decisions
• PR description explains the approach and includes Closes #<issue>

Out of scope

• Distributed/shared breaker state
• Per-endpoint bulkheads

Rewards

Part of the GrantFox OSS / Official Campaign (FWC26) — this task may be rewarded. PR quality is assessed by AI: depth, correctness under edge cases, meaningful tests, and clean design are what earn the reward. Shallow changes (typos, formatting, trivial docs) do not qualify.