fix(sweep): dedup the re-gate fan-out so a burst collapses to one effective sweep - #1297
Merged
Conversation
…ective sweep The ~2-min cron enqueues a fan-out job each tick, but fanOutAgentRegateSweepJobs had no global dedup. When a burst of fan-out jobs ran at once — a deploy-restart cron catch-up, or fan-out jobs that queued behind a heavy per-PR re-review backlog and then drained together — EACH one enqueued a per-repo sweep before the per-repo dispatch-stamp in-flight guard could engage, producing redundant overlapping sweeps (observed ~3x on the metagraphed dry-run). Those redundant sweeps tripled the per-PR load, which delayed the next fan-out, which then burst in turn — a self-sustaining cascade. Add an atomic fan-out dedup: claimRegateFanoutSlot does a conditional UPDATE on the global_agent_controls singleton (new last_regate_fanout_at column, migration 0063) that matches only when the last fan-out is unset or older than the dedup window. D1 serializes writes, so a burst collapses to exactly ONE winner per window; the rest get 0 changes and skip (audited deduped). One effective fan-out per window keeps the per-PR load bounded, which stops the backlog that was delaying subsequent fan-outs — breaking the cascade at its source. Fail-open on a driver error so the fleet never stalls.
Contributor
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1297 +/- ##
=======================================
Coverage 95.08% 95.08%
=======================================
Files 178 178
Lines 20046 20054 +8
Branches 7211 7212 +1
=======================================
+ Hits 19060 19068 +8
Misses 395 395
Partials 591 591
🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Eliminates the two residuals from the metagraphed dry-run: the deploy-restart cron burst → redundant sweeps, and the consequent heavy per-PR re-reviews delaying the fan-out. Both share one root:
fanOutAgentRegateSweepJobshad no global dedup. When a burst of fan-out jobs ran at once (a deploy-restart cron catch-up, or fan-out jobs queued behind a per-PR backlog then drained together), each enqueued a per-repo sweep before the per-repo dispatch-stamp guard (#1294) could engage — ~3× redundant sweeps, which tripled the per-PR load, which delayed the next fan-out, which burst in turn: a self-sustaining cascade.Fix
An atomic fan-out dedup:
claimRegateFanoutSlotdoes a conditionalUPDATEon theglobal_agent_controlssingleton (newlast_regate_fanout_atcolumn, migration 0063) that matches only when the last fan-out is unset or older than the dedup window (SWEEP_FANOUT_DEDUP_MS = 90s, below the 2-min cron cadence). D1 serializes writes, so a burst collapses to exactly one winner per window; the rest get 0 changes and skip (auditeddeduped). One effective fan-out per window keeps the per-PR load bounded — which stops the backlog that was delaying subsequent fan-outs, breaking the cascade at its source. Fail-open on a driver error so the fleet never stalls.Scope
0063_sweep_fanout_marker.sql(one nullable column on the existing singleton),claimRegateFanoutSlot(src/db/repositories.ts),SWEEP_FANOUT_DEDUP_MS(src/settings/agent-sweep.ts), the dedup guard infanOutAgentRegateSweepJobs(src/queue/processors.ts). No new binding.Validation
npm run test:ci— full gate green ·npm audit— 0 vulnerabilities ·db:migrations:check— contiguousdenied. CAS unit tests: first claim wins (NULL marker), claims inside the window lose, a claim past the window wins again; fails open (returns true) on a DB error. Nullable/no-default migration → backward-compatible.v8 ignoreon the?? 0driver-anomaly fallback, matching the existingmarkUnseenOpenIssuesClosedidiom) · rebased onto latestmainSafety
site//CNAME/lovable; no changelog edit