Fix agent should not retry when the required fix needs domain expertise the agent lacks

[fullsend-ai-retro]

What happened

After deboer-tim requested changes and ralphbean issued /fs-fix on PR #1265, four fix agent runs were dispatched between 13:59 and 14:33 UTC on 2026-05-21 (run IDs: 26230699223, 26230995338, 26232380969, 26232666602). Three failed and one succeeded, but the actual fix commits were authored by ralphbean manually. The fix agent consumed compute resources across 4 runs without producing the commits that resolved the review feedback.

What could go better

The fix agent failed repeatedly on what was essentially a domain-knowledge task: understanding which GCP IAM roles are still needed when specific CLI flags are used. The review feedback required understanding the relationship between --inference-wif-provider, --skip-mint-check, --mint-url flags and their GCP IAM requirements. This is moderate-confidence — I cannot see the fix agent's internal traces to know exactly why it failed, but the pattern of 3 failures followed by a human doing the work suggests the agent lacked sufficient context to make the correction. The duplicate /fs-fix from rh-hemartin also triggered unnecessary runs.

Proposed change

Two improvements: (1) Add deduplication logic to the fix dispatch so that if a /fs-fix is already in progress for a PR, subsequent /fs-fix commands within a short window (e.g., 5 minutes) are queued or skipped rather than spawning parallel runs. (2) Consider adding a retry budget to the fix agent — after N consecutive failures on the same PR, post a comment suggesting the fix may require human intervention rather than continuing to consume resources.

Validation criteria

Duplicate /fs-fix commands within a 5-minute window on the same PR should not spawn independent parallel runs. After 2 consecutive fix agent failures on the same PR, the agent should post a comment indicating it could not resolve the feedback and suggesting human intervention. Track fix agent success rate over the next 20 fix runs — the goal is to reduce wasted compute on runs where the agent ultimately cannot produce the fix.

---

Generated by retro agent from #1265

[fullsend-ai-triage]

Triage Summary

This is a well-specified feature request for two improvements to fix agent dispatch:

1. Command deduplication — Prevent duplicate /fs-fix commands within a short window from spawning independent parallel runs.
2. Consecutive failure budget — After N consecutive fix failures on the same PR, post a comment suggesting human intervention instead of continuing to consume resources.

Related Issues

• Add circuit breaker to review-fix feedback loop #902 (circuit breaker for review-fix feedback loop) — related but distinct. That issue addresses automated review→fix loops; this one addresses manual /fs-fix command deduplication and failure budgets.
• Fix agent fails silently when it cannot produce output within iteration limit #976 (fix agent fails silently) — complementary. That issue is about providing feedback on failure; this one is about limiting the number of failures before escalating.

Recommended Approach

The dedup logic likely belongs in the /fs-fix comment handler or the dispatch workflow — check for in-progress fix runs on the same PR before dispatching. The retry budget requires tracking consecutive failures per PR, possibly by querying recent workflow run statuses.

Proposed Test Cases

Test 1 — Deduplication:
  Given a PR with a fix agent run already in progress
  When another /fs-fix is issued within 5 minutes
  Then the second command is skipped (not dispatched)
  And a comment is posted explaining the dedup

Test 2 — Retry budget:
  Given a PR with 2 consecutive fix agent failures
  When another /fs-fix is issued
  Then no new run is dispatched
  And a comment is posted suggesting human intervention

Test 3 — Window expiry:
  Given a PR where the last fix run completed > 5 minutes ago
  When /fs-fix is issued
  Then a new fix agent run is dispatched normally

Severity: Medium — causes wasted compute but does not block core functionality.

---

Labels: Fix agent dispatch deduplication and retry budget feature targeting the dispatch and fix agent subsystems.

[Keesan12]

This is a good example of why retry admission should include capability fit, not just failure count. If the required fix depends on domain expertise the agent lacks, another run is not really a retry; it is repeated guesswork.

I would classify failures after each run as mechanical, context-missing, domain-expertise-needed, blocked-by-reviewer, or transient. Only the first and last categories should auto-admit another fix attempt.

That would also make the final comment more honest: "stopped because domain expertise is required" is much better than four failed runs and a vague handoff.

[Keesan12]

This looks like a domain-fit issue more than a simple retry problem. I would classify each fix failure as mechanical, context-missing, domain-expertise-needed, reviewer-blocked, or transient, and only auto-admit the first and last categories. That would stop repeated guesswork from consuming runs when the fix really needs a human.