Priority: High
Description
RpcClient.call() quarantines an endpoint for 30s on any exception from the callback, without distinguishing network/connectivity failures from application-level errors (malformed XDR, validation errors). Since call() retries up to MAX_RETRIES=3 times on a new endpoint each time, one bad request can cascade-quarantine up to 3 healthy endpoints simultaneously.
Location
engine-bridge/src/rpc-client.ts:30-48 (call)
Current Behavior
try {
const server = new SorobanRpc.Server(ep.url, {...});
return await fn(server);
} catch (err) {
lastError = err;
ep.deadUntil = Date.now() + QUARANTINE_MS; // any error at all
logger.warn(`[RpcClient] ${ep.url} quarantined -- ${(err as Error).message}`);
}Expected Behavior
Only network/transport-level errors (timeouts, connection refused, 5xx) should quarantine an endpoint; application-level errors that will fail identically everywhere should propagate immediately without punishing the endpoint.
Repro / Evidence
Construct RpcClient with 3 URLs, make fn synchronously throw new Error("invalid XDR"); observe all 3 endpoints' deadUntil set after a single call().
Impact
A single malformed/invalid submission can quarantine every configured RPC endpoint for 30s, denying service to all other concurrent callers of the shared RpcClient (used by EventPropagator, HeartbeatMonitor, GasOracle, NonceManager, TxAggregator) — a low-cost, repeatable self-inflicted DoS.
Suggested Fix
Classify errors before quarantining — only quarantine on network/transport failures; rethrow application-level errors immediately without marking the endpoint dead.
Acceptance Criteria
Definition of Done
Priority: High
Description
RpcClient.call()quarantines an endpoint for 30s on any exception from the callback, without distinguishing network/connectivity failures from application-level errors (malformed XDR, validation errors). Sincecall()retries up toMAX_RETRIES=3times on a new endpoint each time, one bad request can cascade-quarantine up to 3 healthy endpoints simultaneously.Location
engine-bridge/src/rpc-client.ts:30-48(call)Current Behavior
Expected Behavior
Only network/transport-level errors (timeouts, connection refused, 5xx) should quarantine an endpoint; application-level errors that will fail identically everywhere should propagate immediately without punishing the endpoint.
Repro / Evidence
Construct
RpcClientwith 3 URLs, makefnsynchronously thrownew Error("invalid XDR"); observe all 3 endpoints'deadUntilset after a singlecall().Impact
A single malformed/invalid submission can quarantine every configured RPC endpoint for 30s, denying service to all other concurrent callers of the shared
RpcClient(used byEventPropagator,HeartbeatMonitor,GasOracle,NonceManager,TxAggregator) — a low-cost, repeatable self-inflicted DoS.Suggested Fix
Classify errors before quarantining — only quarantine on network/transport failures; rethrow application-level errors immediately without marking the endpoint dead.
Acceptance Criteria
deadUntil.Definition of Done