Skip to content

fix(sweep): dedup the re-gate fan-out so a burst collapses to one effective sweep - #1297

Merged
JSONbored merged 1 commit into
mainfrom
fix/fanout-dedup
Jun 25, 2026
Merged

fix(sweep): dedup the re-gate fan-out so a burst collapses to one effective sweep#1297
JSONbored merged 1 commit into
mainfrom
fix/fanout-dedup

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

Eliminates the two residuals from the metagraphed dry-run: the deploy-restart cron burst → redundant sweeps, and the consequent heavy per-PR re-reviews delaying the fan-out. Both share one root: fanOutAgentRegateSweepJobs had no global dedup. When a burst of fan-out jobs ran at once (a deploy-restart cron catch-up, or fan-out jobs queued behind a per-PR backlog then drained together), each enqueued a per-repo sweep before the per-repo dispatch-stamp guard (#1294) could engage — ~3× redundant sweeps, which tripled the per-PR load, which delayed the next fan-out, which burst in turn: a self-sustaining cascade.

Fix

An atomic fan-out dedup: claimRegateFanoutSlot does a conditional UPDATE on the global_agent_controls singleton (new last_regate_fanout_at column, migration 0063) that matches only when the last fan-out is unset or older than the dedup window (SWEEP_FANOUT_DEDUP_MS = 90s, below the 2-min cron cadence). D1 serializes writes, so a burst collapses to exactly one winner per window; the rest get 0 changes and skip (audited deduped). One effective fan-out per window keeps the per-PR load bounded — which stops the backlog that was delaying subsequent fan-outs, breaking the cascade at its source. Fail-open on a driver error so the fleet never stalls.

Scope

  • Migration 0063_sweep_fanout_marker.sql (one nullable column on the existing singleton), claimRegateFanoutSlot (src/db/repositories.ts), SWEEP_FANOUT_DEDUP_MS (src/settings/agent-sweep.ts), the dedup guard in fanOutAgentRegateSweepJobs (src/queue/processors.ts). No new binding.

Validation

  • npm run test:ci — full gate green · npm audit — 0 vulnerabilities · db:migrations:check — contiguous
  • Invariant: a burst of fan-outs collapses to one — the second claims nothing and audits denied. CAS unit tests: first claim wins (NULL marker), claims inside the window lose, a claim past the window wins again; fails open (returns true) on a DB error. Nullable/no-default migration → backward-compatible.
  • Every changed line + branch covered (one honest v8 ignore on the ?? 0 driver-anomaly fallback, matching the existing markUnseenOpenIssuesClosed idiom) · rebased onto latest main

Safety

  • Global infra, not repo-specific — no hard-coded repo names. No secrets / wallets / hotkeys / trust scores / reward values; no site/ / CNAME / lovable; no changelog edit

…ective sweep

The ~2-min cron enqueues a fan-out job each tick, but fanOutAgentRegateSweepJobs
had no global dedup. When a burst of fan-out jobs ran at once — a deploy-restart
cron catch-up, or fan-out jobs that queued behind a heavy per-PR re-review
backlog and then drained together — EACH one enqueued a per-repo sweep before the
per-repo dispatch-stamp in-flight guard could engage, producing redundant
overlapping sweeps (observed ~3x on the metagraphed dry-run). Those redundant
sweeps tripled the per-PR load, which delayed the next fan-out, which then burst
in turn — a self-sustaining cascade.

Add an atomic fan-out dedup: claimRegateFanoutSlot does a conditional UPDATE on
the global_agent_controls singleton (new last_regate_fanout_at column, migration
0063) that matches only when the last fan-out is unset or older than the dedup
window. D1 serializes writes, so a burst collapses to exactly ONE winner per
window; the rest get 0 changes and skip (audited deduped). One effective fan-out
per window keeps the per-PR load bounded, which stops the backlog that was
delaying subsequent fan-outs — breaking the cascade at its source. Fail-open on a
driver error so the fleet never stalls.
@dosubot dosubot Bot added the size:M label Jun 25, 2026
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@JSONbored JSONbored self-assigned this Jun 25, 2026
@codecov

codecov Bot commented Jun 25, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.08%. Comparing base (398d942) to head (6ad4a6b).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1297   +/-   ##
=======================================
  Coverage   95.08%   95.08%           
=======================================
  Files         178      178           
  Lines       20046    20054    +8     
  Branches     7211     7212    +1     
=======================================
+ Hits        19060    19068    +8     
  Misses        395      395           
  Partials      591      591           
Files with missing lines Coverage Δ
src/db/repositories.ts 96.09% <100.00%> (+0.01%) ⬆️
src/queue/processors.ts 87.23% <100.00%> (+0.03%) ⬆️
src/settings/agent-sweep.ts 100.00% <100.00%> (ø)
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@JSONbored
JSONbored merged commit 4fa5a89 into main Jun 25, 2026
19 checks passed
@JSONbored
JSONbored deleted the fix/fanout-dedup branch June 25, 2026 02:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant