Repository navigation
fix(coordinator): retry storage failures and prepare grant redaction - #2847
Open
vincentkoc wants to merge 4 commits into
Open
vincentkoc wants to merge 4 commits into
vincentkoc wants to merge 4 commits into
Conversation
vincentkoc
force-pushed
the
fix/coordinator-storage-reader-rollout-20261015
branch
from
October 15, 2026 03:17
fcf8120 to
4b64315
Compare
vincentkoc
marked this pull request as ready for review
October 15, 2026 03:18
Contributor
|
ClawSweeper reviews are opt-in for maintainers. If a review is required, finish the work and validation first, then comment |
vincentkoc
force-pushed
the
fix/coordinator-storage-reader-rollout-20261015
branch
from
October 15, 2026 03:21
a74093e to
7edfe61
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What Problem This Solves
Fixes missing retry signals when transient coordinator storage errors are caught internally, and prepares a safe migration for an encrypted GitHub grant header visible in Cloudflare invocation logs.
User Impact
Transient Cloudflare storage failures return a retryable response without replaying a possibly committed operation. Provider error responses and Node runtime errors retain their existing behavior. Pairing and bridge authentication accept both header names while stage A still writes only the legacy header.
The observed header contained encrypted grant ciphertext. This is not a claim that plaintext credentials were logged. Redaction is completed only by the separately gated stage B deployment.
Why This Change Was Made
The inner Fleet catch converted storage failures to HTTP 500 before the Worker boundary could classify them. The Cloudflare runtime now records failures at actual storage operations, preserving each original exception. Transaction callback errors are not misclassified as storage failures. This matters because provider errors also use retryability flags.
A shared grant reader accepts both names and prefers the legacy value, preventing a caller-supplied new-name value forwarded by an old Worker from shadowing the authoritative grant. Stage A strips that new-name input and retains the legacy-only writer.
Commit sequence:
658dba4b1f3ef3a774aff1b0d86e8c460f996dad— storage reliability repair.b92b0a4c67b33d04bf212ecccc4e3ad399ff8857— compatible readers, legacy-only writer.4b643154614c34f6dab664490eac7057591570c2— preserve provider errors while classifying native storage failures.7edfe614be987daf6bfb59082fa721a1d5b234cc— refresh the existing AWS RDS trust-bundle checksum required by the Node container build.Production gate: deploy the squash-merged A revision and record its source SHA and version at 100%. The Worker receipt alone does not prove Fleet Durable Object convergence; verify an actual Fleet invocation runs the A version. Do not advance to B without that evidence. Preserve compatible rollback targets. See
docs/operations.mdfor the sequence.Evidence
At
4b643154614c34f6dab664490eac7057591570c2, whose runtime and tests are unchanged by the certificate-pin commit: the complete Worker suite passed 4,473 tests, with 15 skipped (90 test files passed, two skipped). Worker and Node TypeScript checks, touched lint/format checks andgit diff --checkpassed.Coverage includes both grant header forms through both actual consumers, authoritative legacy precedence, forged-header stripping, storage provenance through transactions and alarms, preservation of error identity, no operation replay, provider errors with the same flags, and unchanged Node error behavior. The full suite also preserves provider cleanup/cancellation and uncertain-commit recovery assertions.
The Node container build exposed an outdated pin for AWS's official RDS bundle. Two HTTPS downloads matched SHA-256
90a6f1a7978a1b1b096dca6c57511ca48d3b8cf052f8e704baf358f34ea2f21b; 114 unique Amazon RDS CA certificates parsed and were unexpired. Only the checksum, its matching test and changelog changed. Seven Node/Fargate contract tests passed. PostgreSQL-only trust scope and checksum enforcement remain intact.Independent source review is required at the final SHA; hosted checks must pass before merge. Remaining live proof: the merged A deployment receipt and actual Fleet Durable Object convergence. No deployment has been performed for this change.