What happened
The first review agent run on PR #816 (run 25687001021) failed with exit code 1, and 3 more review runs failed later. The PR author received a "fullsend review is working on this" comment but no follow-up indicating the review failed. The first successful bot review didn't arrive until ~3 hours later.
The failures left no trace on the PR itself — the author and reviewer had to rely on the human review that arrived at 20:04 and had no way to know whether the bot would eventually weigh in. The failure details were locked in GitHub Actions artifacts that returned HTTP 403 when we attempted to access them during this retro.
What could go better
When a review agent run fails, the post-review script (or a fallback in the harness) should post a brief comment on the PR indicating that the automated review failed, with a link to the Actions run for debugging. This prevents the ambiguous state where the PR shows "fullsend review is working on this" but never follows up.
Confidence: Medium-High. The "working on this" comment with no follow-up is clearly a gap. The fix is straightforward — post a failure comment. However, I haven't inspected the post-review script to confirm whether it already handles failures and this was an edge case, or whether failure notification is entirely missing.
A secondary concern: the artifacts for failed runs returned 403. This may be an artifact retention or access policy issue that makes debugging difficult even when someone does investigate.
Proposed change
In the review harness post-script (scripts/post-review.sh in the .fullsend repo, scaffolded from internal/scaffold/fullsend-repo/scripts/), add a failure path:
- If the review agent exits with a non-zero code and no review result was produced, post a PR comment: "Automated review failed ([run link]). A human review is still required."
- If the agent produced a partial result, still post the failure comment but include any findings that were captured.
- Update or remove the "working on this" sticky comment to reflect the failure state.
This is a harness-level change that would benefit all repos using fullsend review.
Validation criteria
On the next 5 review agent failures across any fullsend-managed repo, verify that a failure comment is posted on the PR within 2 minutes of the run completing. The comment should include a link to the Actions run. The "working on this" comment should be updated or removed.
Generated by retro agent from #816
What happened
The first review agent run on PR #816 (run 25687001021) failed with exit code 1, and 3 more review runs failed later. The PR author received a "fullsend review is working on this" comment but no follow-up indicating the review failed. The first successful bot review didn't arrive until ~3 hours later.
The failures left no trace on the PR itself — the author and reviewer had to rely on the human review that arrived at 20:04 and had no way to know whether the bot would eventually weigh in. The failure details were locked in GitHub Actions artifacts that returned HTTP 403 when we attempted to access them during this retro.
What could go better
When a review agent run fails, the post-review script (or a fallback in the harness) should post a brief comment on the PR indicating that the automated review failed, with a link to the Actions run for debugging. This prevents the ambiguous state where the PR shows "fullsend review is working on this" but never follows up.
Confidence: Medium-High. The "working on this" comment with no follow-up is clearly a gap. The fix is straightforward — post a failure comment. However, I haven't inspected the post-review script to confirm whether it already handles failures and this was an edge case, or whether failure notification is entirely missing.
A secondary concern: the artifacts for failed runs returned 403. This may be an artifact retention or access policy issue that makes debugging difficult even when someone does investigate.
Proposed change
In the review harness post-script (
scripts/post-review.shin the.fullsendrepo, scaffolded frominternal/scaffold/fullsend-repo/scripts/), add a failure path:This is a harness-level change that would benefit all repos using fullsend review.
Validation criteria
On the next 5 review agent failures across any fullsend-managed repo, verify that a failure comment is posted on the PR within 2 minutes of the run completing. The comment should include a link to the Actions run. The "working on this" comment should be updated or removed.
Generated by retro agent from #816