feat(transfers): make transfer creation idempotent under retries - #135
Open
leocagli wants to merge 1 commit into
Open
feat(transfers): make transfer creation idempotent under retries#135leocagli wants to merge 1 commit into
leocagli wants to merge 1 commit into
Conversation
createTransfer submitted the payment to the provider before persisting anything, so a client or provider retry that arrived after a timeout found no record of the first attempt and moved money a second time. Each attempt also wrote its own audit entry, which made the duplicate hard to see afterwards. The fix reserves the (actor, key) pair before any side effect runs, and stores the terminal result once the transfer exists: begin -> reserve, replay, or refuse provider call and persistence complete -> store the result for later retries release -> on failure, so a correct retry can still win Design notes: - Scoped by API token. Keys are chosen by clients, so a bare namespace lets one caller collide with, or probe for, another operation that is not theirs. - The fingerprint covers the fields that determine the transfer, not the raw body, and is canonicalized: property order, an amount sent as a string and unrelated extra fields do not turn a legitimate retry into a 409. - Conflicting reuse is checked before in-flight state, so a client reusing a key for a different operation gets the same clear answer either way instead of being sent into a retry loop that can never win. - A replay short-circuits before the quote is recomputed. Rates move, so recomputing would hand back a different transfer under the same key. - A provider failure releases the reservation. Burning the key would be worse than the duplicate it prevents. Compatibility: the header is now required on POST /api/transfers, which is a breaking change for clients that omit it. That is deliberate on a money path: a client with no key is not opting out of protection, it is unaware it needs it. The service entry point keeps the context optional because idempotency is actor-scoped and internal callers have no actor to scope to; the route supplies it on every request-driven creation. Tests: 28 new (19 service, 9 HTTP) covering retry, conflict, concurrency, provider failure and recovery, restart, actor scoping and fingerprint canonicalization. The concurrency case re-enters from inside the provider call, which reproduces the exact window the bug lived in rather than simulating it. Suite goes 153 to 181, all passing. Two existing test files were updated to send the header; no assertion was relaxed. Closes RemitFlow#128 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #128
The failure mode
createTransfercalled the provider before persisting anything:A retry that arrived in that window found no record of the first attempt and moved money again. Each attempt also wrote its own
transfer.createdaudit entry, so the duplicate was hard to see afterwards.The shape of the fix
The
(actor, key)pair is reserved before any side effect runs, and the terminal result is stored once the transfer exists:begincompletereleaseReserving first is the whole point. Any design that writes the record after the provider call has the same window the bug lived in, just narrower.
Design decisions worth reviewing
Scoped by API token. Keys are chosen by clients, so a bare key namespace lets one caller collide with, or probe for, an operation that is not theirs. Two tests cover this.
The fingerprint covers the fields that determine the transfer, not the raw body. An unrelated extra property must not read as a conflicting retry, and
amountis normalized because"100"and100both validate and produce the same transfer. Property order is canonicalized at every depth, since JSON key order is not semantically meaningful and a client rebuilding a retry from a hash map will not preserve it. Treating any of those as a 409 would reject legitimate retries.Conflicting reuse is checked before in-flight state. If a key is reused for a different operation, the client gets the same clear answer whether or not the first call has finished. Reporting "still in progress" for what is really a client bug sends them into a retry loop that can never win.
A replay short-circuits before the quote is recomputed. Rates move. Recomputing would hand the client a different transfer under the same key, which is the duplicate this is meant to prevent.
A provider failure releases the reservation. Burning the key would be worse than the duplicate it prevents: the client retries correctly, with the same key, and gets "still in progress" forever even though nothing is in progress.
Compatibility impact
Idempotency-Keyis now required onPOST /api/transfers. This breaks clients that omit it, and that is deliberate on a money path: a client with no key is not opting out of protection, it is unaware it needs it. Rejecting with 400 is the only outcome that cannot silently duplicate a transfer.The service entry point keeps the context optional. That is not a loophole, it is what "actor-scoped" implies: an actor exists at the HTTP boundary (
req.token) and internal callers have none to scope to. The route supplies the context on every request-driven creation, and a test asserts the route rejects a missing header.Tests
28 new, 19 at the service level and 9 over HTTP.
The concurrency case is the one I would look at first. The service is synchronous, so two requests cannot interleave on their own; the test re-enters
createTransferfrom inside the provider call, which reproduces the exact window the bug lived in rather than simulating one:Coverage: retry replays without duplicate movement, exactly one provider command and one audit record across three attempts, conflicting payload gives 409, concurrent attempt gives 409, conflict beats race in the ordering, provider failure releases the key and the retry succeeds, a completed key survives a later outage, restart clears reservations and transfers together, same key from a different actor stays separate, and the fingerprint canonicalization cases.
The regression assertion for the original bug is
providerCalls === 1after three retries. Against the old code that is 3.Verification
Suite goes from 153 to 181.
test/requireScope.test.jsandtest/moneyPrecision.test.jswere updated to send the header on theirPOST /api/transferscalls, each with its own key. No assertion was relaxed, nothing was skipped, and a shared key was deliberately avoided since it would have made the second create replay the first and quietly void those tests.Not done here
Reservation records inherit the store's durability, which for the in-memory Map that backs this demo means a restart clears them. They live in the same store as the transfers on purpose, so the two clear together: a surviving reservation would replay a transfer that no longer exists. Moving to a real datastore makes both durable without touching this logic, which is why I did not add a separate persistence path for it.