Normalize trigger_source in the harness, not the LLM

[fullsend-ai-retro]

What happened

The fix agent failed schema validation in 5 of 8 runs (May 15-21) because it wrote the raw TRIGGER_SOURCE username into fix-result.json instead of the enum "bot" or "human". Issue #1300 identified this pattern. PR #1308 fixed it by adding explicit mapping instructions to agents/fix.md and skills/fix-review/SKILL.md, telling the LLM to check if the username ends in [bot] and map accordingly.

What could go better

The fix relies on the LLM consistently following normalization instructions — the same approach that failed in the first place (the agent definition already implied the mapping, just not explicitly enough). The harness scripts (pre-fix.sh, post-fix.sh) already contain is_bot_user() functions that perform this exact check. If the harness passed a pre-normalized TRIGGER_TYPE env var ("bot" or "human") alongside the raw TRIGGER_SOURCE username, the LLM would just copy the value — no interpretation needed. This eliminates the failure class rather than mitigating it with better instructions.

Confidence: High. The is_bot_user() function already exists in two harness scripts, and the workflow's reusable-dispatch.yml already validates the username format. Adding a derived env var is a small, safe change.

Proposed change

In internal/scaffold/fullsend-repo/harness/fix.yaml, add a new runner_env variable TRIGGER_TYPE that is pre-normalized. This requires a small change in the workflow (reusable-fix.yml or pre-fix.sh) to derive the value using the existing is_bot_user() logic before the agent starts. Then update agents/fix.md and skills/fix-review/SKILL.md to instruct the agent to use TRIGGER_TYPE directly as the trigger_source value in fix-result.json, rather than performing its own normalization of TRIGGER_SOURCE. Keep TRIGGER_SOURCE (the raw username) available for display/logging purposes.

Validation criteria

Schema validation failures on the trigger_source field should drop to zero. The next 10 fix agent runs should not fail due to trigger_source enum mismatch. Verify by checking gh run list --repo fullsend-ai/.fullsend --workflow=fix.yml --limit 10 after the change is deployed.

---

Generated by retro agent from #1308

[fullsend-ai-triage]

Triage Summary

This retro-generated issue is well-specified and ready for implementation.

Problem: The fix agent's trigger_source schema validation failures were mitigated by PR #1308 (better LLM instructions), but the root cause remains — the LLM is asked to interpret a raw username into an enum value, which is inherently fragile.

Proposed fix: Pre-normalize a TRIGGER_TYPE env var ("bot" or "human") in the harness using the existing is_bot_user() logic, so the agent copies the value directly instead of deriving it.

Key files to change:

• internal/scaffold/fullsend-repo/harness/fix.yaml — add TRIGGER_TYPE runner_env
• pre-fix.sh or reusable-fix.yml — derive the value using is_bot_user()
• agents/fix.md and skills/fix-review/SKILL.md — update instructions to use TRIGGER_TYPE

Validation: The next 10 fix agent runs should show zero trigger_source schema validation failures.

Test case: TRIGGER_TYPE normalization
- Input: TRIGGER_SOURCE="dependabot[bot]" → TRIGGER_TYPE should be "bot"
- Input: TRIGGER_SOURCE="octocat" → TRIGGER_TYPE should be "human"
- fix-result.json trigger_source field should equal TRIGGER_TYPE verbatim
- Schema validation should pass for all runs

---

Labels: Retro-generated improvement for the fix agent's harness scaffold, medium priority since a workaround (PR #1308) is already merged.

[ifireball]

/fs-prioritize

[fullsend-ai-prioritize]

RICE Priority Score: 2.7

Score breakdown

Dimension	Score	Reasoning
Reach	1.5	The fix agent runs across all users of the platform, and 5 of 8 runs failed schema validation on trigger_source (May 15-21). While a workaround (PR #1308) is already merged, the fragile LLM-based normalization still affects all fix agent consumers, making this a multi-customer concern.
Impact	1	Medium impact — a workaround already exists via PR #1308's improved instructions, so the failure class is mitigated but not eliminated. This change would make the fix agent more reliable by removing an inherently fragile LLM interpretation step, but it's not currently blocking workflows.
Confidence	0.9	Very high confidence — the issue is exceptionally well-specified with exact files to change, existing helper functions to reuse (is_bot_user()), clear validation criteria, and a triage comment confirming the approach. The scope is narrow and well-understood.
Effort	0.5	Simple change — the is_bot_user() function already exists in two harness scripts, the workflow already validates the username format, and the change involves adding one derived env var and updating instructions in 2-3 files. Well-scoped with minimal risk.

Formula: (1.5 x 1 x 0.9) / 0.5 = 2.7