Skip to content

board reports PRs that were never created — outcome capture substring-matches command text, counting mentions not results #1134

Description

@kokevidaurre

The outcome columns on an execution row (prs_created, issues_created, commits) are derived by substring-matching the command text of Bash tool calls. That counts mentions, not creations — so the numbers on squads board (and every surface reading the ledger) can be pure fiction.

Repro from a real run (2026-07-15)

A lane finished with prs_created: 2, issues_created: 1 on its ledger row. It created zero PRs and zero issues — its git credentials had expired mid-run (#1133), so every push and gh call failed with Bad credentials. The PR that eventually landed its work was opened by hand, hours later.

Where the phantom "2 PRs" came from, verbatim from the transcript:

  1. grep -rln "tool_use\|gh pr create\|git commit" --include="*.py" . — a search for the string.
  2. git commit -m "…" whose message documents the patterns gh pr create / gh issue create / git commit.

The lane was implementing the very feature that counts PR creations, so the counter counted its own source code and commit message.

Why it matters

  • squads board reports "N PRs created · today" as a headline number. Today it says 2; the true count from that run is 0. A metric that is silently wrong is worse than one that is absent — it launders an intent (or a mere mention) into a result.
  • Failure makes it worse, not better: a lane whose pushes all fail still increments the counter, so the number climbs precisely when nothing is shipping.
  • This is the ledger-side capture; the API-side derivation for harness sessions shares the same approach and needs the same fix.

Fix direction

Count the result, not the command. gh pr create prints the PR URL on success and exits non-zero on failure — so parse the tool_result for https://…/pull/<n> (likewise /issues/<n>), and take git commit from the resulting SHA rather than the invocation. That yields verifiable identifiers instead of a guess, makes the numbers dedupable across sources, and drops to 0 when nothing shipped.

If a verified count isn't available for a source, render nothing — not a zero, and not an attempt count dressed as a result.

Acceptance

  • A run whose gh pr create fails (bad credentials, CI reject, anything) reports 0 PRs.
  • A run that greps for, echoes, or commits the literal text gh pr create reports 0 PRs.
  • A run that actually opens 2 PRs reports 2, and the row carries their numbers/URLs.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions