Skip to content

fix(agents): surface stdout-only CLI errors - #202

Open
jasonqlwilliams-alt wants to merge 1 commit into
kunchenguid:mainfrom
jasonqlwilliams-alt:fm/gnhf-shared-stderr-defect-s2
Open

fix(agents): surface stdout-only CLI errors#202
jasonqlwilliams-alt wants to merge 1 commit into
kunchenguid:mainfrom
jasonqlwilliams-alt:fm/gnhf-shared-stderr-defect-s2

Conversation

@jasonqlwilliams-alt

Copy link
Copy Markdown
Contributor

Intent

Fix the shared child-process error-reporting defect in src/core/agents/stream-utils.ts used by the Codex, Copilot, and Pi adapters. Build on the established PR #190 behavior: a non-zero CLI exit may carry the real error on stdout while stderr is empty. Include a bounded stdout tail when stderr alone lacks the error, parse structured JSON error fields, classify permanent errors from the relevant combined output, and clearly report when both streams are empty. Require behavioral adapter coverage for Codex, Copilot, and Pi stdout-only non-zero exits. Reuse the corrected shared helper for Claude only if it reduces duplication without broadening behavior or risk. Remain compatible with the fully green Pi JSON recovery PR #195 and do not modify its branch. Deliver a review-ready upstream contribution PR with all canonical checks green. The captain explicitly authorizes unattended --yes validation of preserved commit 055f17b46bb6709f42af5e93220561baee792808 under no-mistakes v1.48.0. Do not merge, publish, deploy, change policy, waive checks, alter credentials, reset, force, or discard preserved work.

What Changed

  • Capture a bounded stdout tail for non-zero agent exits, extract structured JSON errors, and fall back to truncated raw output or an explicit no-output message.
  • Share exit reporting with Claude while keeping permanent-error classification limited to stderr and structured stdout error fields.
  • Add stdout-only failure coverage for the Codex, Copilot, and Pi adapters, plus shared handler coverage for empty output.

Risk Assessment

✅ Low: The change is well-bounded, preserves the established Claude semantics, fixes all shared adapter paths, and adds behavior-level coverage for each required adapter without conflicting with Pi JSON recovery.

Testing

Focused shared-helper and adapter tests, the Claude stdout/empty-stream regression, and built-CLI stdout-only failures for Codex, Copilot, and Pi all passed; after correcting and cleaning an initial harness working-directory mistake, the final transcript directly demonstrates the user-visible CLI behavior. No screenshot was needed because this changes process-error text rather than graphical layout.

Evidence: Stdout-only adapter failure transcript

Codex, Copilot, and Pi child processes each exited 1 with zero stderr bytes and a structured stdout error. The built gnhf CLI displayed and persisted the exact error for every adapter.

=== codex: child CLI stimulus ===
exit code: 1
stderr bytes: 0
stdout:
{"type":"result","subtype":"error_during_execution","is_error":true,"result":"Invalid model name: claude-nonexistent-5"}

=== codex: gnhf end-user terminal result ===
                                                                                
╭───────────────────────────────────────────────────────────────────────────────────────────────╮
│ × gnhf stopped                                                                                │
│   codex ran for 0s before: codex exited with code 1: Invalid model name: claude-nonexistent-5 │
╰───────────────────────────────────────────────────────────────────────────────────────────────╯

  iterations      1 total       0 good       1 failed
  tokens          0 in          0 out        
  branch diff     0 commits     +0           -0
  files           0 added       0 updated    0 deleted

  notes           /tmp/no-mistakes-evidence/01KZVSXH8QZZFSDG7YNWDKGHY6/codex-repo/.gnhf/runs/surface-codex-stdout-e8eab2/notes.md
  debug log       /tmp/no-mistakes-evidence/01KZVSXH8QZZFSDG7YNWDKGHY6/codex-repo/.gnhf/runs/surface-codex-stdout-e8eab2/gnhf.log

  next steps      git log --oneline 1a5fdba75199..HEAD
                  git diff --stat 1a5fdba75199..HEAD
                  gh pr create

  too much        git push no-mistakes:
  to review?      https://github.com/kunchenguid/no-mistakes

=== codex: persisted notes.md ===
**Summary:** [ERROR] codex exited with code 1: Invalid model name: claude-nonexistent-5

=== copilot: child CLI stimulus ===
exit code: 1
stderr bytes: 0
stdout:
{"type":"result","subtype":"error_during_execution","is_error":true,"result":"Invalid model name: claude-nonexistent-5"}

=== copilot: gnhf end-user terminal result ===
                                                                                
╭───────────────────────────────────────────────────────────────────────────────────────────────────╮
│ × gnhf stopped                                                                                    │
│   copilot ran for 0s before: copilot exited with code 1: Invalid model name: claude-nonexistent-5 │
╰───────────────────────────────────────────────────────────────────────────────────────────────────╯

  iterations      1 total       0 good       1 failed
  tokens          0 in          0 out        
  branch diff     0 commits     +0           -0
  files           0 added       0 updated    0 deleted

  notes           /tmp/no-mistakes-evidence/01KZVSXH8QZZFSDG7YNWDKGHY6/copilot-repo/.gnhf/runs/surface-copilot-stdo-5c9af3/notes.md
  debug log       /tmp/no-mistakes-evidence/01KZVSXH8QZZFSDG7YNWDKGHY6/copilot-repo/.gnhf/runs/surface-copilot-stdo-5c9af3/gnhf.log

  next steps      git log --oneline 1a5fdba75199..HEAD
                  git diff --stat 1a5fdba75199..HEAD
                  gh pr create

  too much        git push no-mistakes:
  to review?      https://github.com/kunchenguid/no-mistakes

=== copilot: persisted notes.md ===
**Summary:** [ERROR] copilot exited with code 1: Invalid model name: claude-nonexistent-5

=== pi: child CLI stimulus ===
exit code: 1
stderr bytes: 0
stdout:
{"type":"result","subtype":"error_during_execution","is_error":true,"result":"Invalid model name: claude-nonexistent-5"}

=== pi: gnhf end-user terminal result ===
                                                                                
╭─────────────────────────────────────────────────────────────────────────────────────────╮
│ × gnhf stopped                                                                          │
│   pi ran for 0s before: pi exited with code 1: Invalid model name: claude-nonexistent-5 │
╰─────────────────────────────────────────────────────────────────────────────────────────╯

  iterations      1 total       0 good       1 failed
  tokens          0 in          0 out        
  branch diff     0 commits     +0           -0
  files           0 added       0 updated    0 deleted

  notes           /tmp/no-mistakes-evidence/01KZVSXH8QZZFSDG7YNWDKGHY6/pi-repo/.gnhf/runs/surface-pi-stdout-fa-fef76d/notes.md
  debug log       /tmp/no-mistakes-evidence/01KZVSXH8QZZFSDG7YNWDKGHY6/pi-repo/.gnhf/runs/surface-pi-stdout-fa-fef76d/gnhf.log

  next steps      git log --oneline 1a5fdba75199..HEAD
                  git diff --stat 1a5fdba75199..HEAD
                  gh pr create

  too much        git push no-mistakes:
  to review?      https://github.com/kunchenguid/no-mistakes

=== pi: persisted notes.md ===
**Summary:** [ERROR] pi exited with code 1: Invalid model name: claude-nonexistent-5

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

✅ **Review** - passed

✅ No issues found.

✅ **Test** - passed

✅ No issues found.

  • pnpm exec vitest run src/core/agents/stream-utils.test.ts src/core/agents/codex.test.ts src/core/agents/copilot.test.ts src/core/agents/pi.test.ts src/core/agents/claude.test.ts
  • pnpm run build
  • Built CLI runs in isolated repositories using node dist/cli.mjs "surface <agent> stdout failure" --agent <codex|copilot|pi> --max-iterations 1 --prevent-sleep off
  • Direct child-process checks confirming exit code 1, structured JSON on stdout, and zero stderr bytes for Codex, Copilot, and Pi fixtures
  • pnpm exec vitest run e2e/e2e.test.ts -t "surfaces the claude CLI's own failure text"
  • Verified the terminal summaries and persisted notes.md errors for all three required adapters
  • Initial CLI evidence attempt used the wrong working directory; its results were discarded, and all resulting branch, metadata, build, and dependency artifacts were removed
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant