Skip to content

fix: /ship CHANGELOG and PR body now cover all branch commits (v0.12.4.0)#535

Merged
garrytan merged 3 commits intomainfrom
garrytan/ship-full-commit-coverage
Mar 26, 2026
Merged

fix: /ship CHANGELOG and PR body now cover all branch commits (v0.12.4.0)#535
garrytan merged 3 commits intomainfrom
garrytan/ship-full-commit-coverage

Conversation

@garrytan
Copy link
Copy Markdown
Owner

@garrytan garrytan commented Mar 26, 2026

Summary

CHANGELOG/PR coverage bug fix. When shipping a branch with many commits spanning multiple themes (e.g., performance + dead code removal + test infra), the CHANGELOG and PR summary biased toward recent commits and silently dropped earlier work. Found on the garrytan/post-gen-perf branch where 8 of 12 commits were missing from the PR body.

Two changes:

  • Step 5 (CHANGELOG generation): Restructured to force explicit commit enumeration, theme grouping, and a cross-check step. The model must list all commits, group by theme, write the entry, then verify every commit maps to a bullet.
  • Step 8 (PR body summary): Changed from "bullet points from CHANGELOG" to explicit commit-by-commit coverage. Groups commits into logical sections. Excludes the VERSION/CHANGELOG metadata commit.

Test Coverage

No application code paths changed. This is a prompt template text fix (ship/SKILL.md.tmpl and its generated ship/SKILL.md). Existing bun test suite passes (skill validation, gen-skill-docs quality checks).

A P3 TODO was added for a multi-commit E2E eval that would exercise this failure mode.

Pre-Landing Review

No issues found. Pure markdown/prose changes to skill templates.

TODOS

  • Added: "Multi-commit CHANGELOG completeness eval" (P3) — periodic E2E that creates a 5+ commit branch spanning 3+ themes and verifies CHANGELOG covers all of them.

Test plan

  • bun test passes (skill validation + gen-skill-docs quality)
  • Generated ship/SKILL.md matches template changes
  • Merge conflict with v0.12.3.0 (voice directive PR) resolved cleanly — bumped to v0.12.4.0

🤖 Generated with Claude Code

garrytan and others added 3 commits March 26, 2026 17:31
Step 5 (CHANGELOG generation) restructured to force explicit commit
enumeration, theme grouping, and cross-check before writing. Step 8
(PR body) changed from "bullet points from CHANGELOG" to commit-by-commit
coverage with logical sections. Fixes recency bias that dropped early
commits on long branches.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
CHANGELOG.md conflict: both sides used v0.12.3.0. Kept main's voice
directive entry as v0.12.3.0, bumped our commit coverage fix to v0.12.4.0.

Regenerated all SKILL.md files to incorporate both our ship template
changes (Step 5/8 commit coverage) and main's voice directive feature
(preamble.ts resolver additions).
@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 26, 2026

E2E Evals: ✅ PASS

9/9 tests passed | $.75 total cost | 12 parallel runners

Suite Result Status Cost
e2e-qa-workflow 1/1 $0.08
e2e-review 2/2 $0.33
e2e-workflow 2/2 $0.15
llm-judge 2/2 $0.04
e2e-workflow 2/2 $0.15

12x ubicloud-standard-2 (Docker: pre-baked toolchain + deps) | wall clock ≈ slowest suite

@garrytan garrytan changed the title fix: /ship CHANGELOG and PR body now cover all branch commits (v0.12.3.0) fix: /ship CHANGELOG and PR body now cover all branch commits (v0.12.4.0) Mar 26, 2026
@garrytan garrytan merged commit de20228 into main Mar 26, 2026
18 checks passed
KimYoungHwan8750 pushed a commit to KimYoungHwan8750/gstack-ko that referenced this pull request Mar 27, 2026
…4.0) (garrytan#535)

* fix: /ship CHANGELOG and PR body now cover all branch commits

Step 5 (CHANGELOG generation) restructured to force explicit commit
enumeration, theme grouping, and cross-check before writing. Step 8
(PR body) changed from "bullet points from CHANGELOG" to commit-by-commit
coverage with logical sections. Fixes recency bias that dropped early
commits on long branches.

* chore: bump version and changelog (v0.12.3.0)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
rapidstartup pushed a commit to rapidstartup/gstack that referenced this pull request Mar 29, 2026
…4.0) (garrytan#535)

* fix: /ship CHANGELOG and PR body now cover all branch commits

Step 5 (CHANGELOG generation) restructured to force explicit commit
enumeration, theme grouping, and cross-check before writing. Step 8
(PR body) changed from "bullet points from CHANGELOG" to commit-by-commit
coverage with logical sections. Fixes recency bias that dropped early
commits on long branches.

* chore: bump version and changelog (v0.12.3.0)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant