Skip to content

Release: an (unreleased) CHANGELOG section survives the changelog step whole, by design [PROTECTED: owner merges] - #829

Merged
jayden1711 merged 7 commits into
mainfrom
issue-811
Oct 20, 2026
Merged

jayden1711 merged 7 commits into
mainfrom
issue-811

Conversation

@jayden1711

@jayden1711 jayden1711 commented Oct 19, 2026 •

Copy link
Copy Markdown
Owner

Intent: the changelog step (release.mjs changelog X.Y.Z) never drops a hand-written note. withSection in scripts/lib/changelog.mjs keeps an unreleased section whole, every line under any heading name (the generated ones included). It renames the section's heading to the dated one and writes the generated lists after the notes. So #809's layout rule (end the notes with an empty generated heading) is no longer needed. After round 2, the step also checks its own result: if any hand-written line would be lost, it throws and writes nothing.

Risk class: protected. scripts/** is release tooling, and the owner merges.

Closes #811

What changed

  • scripts/lib/changelog.mjs withSection:
    • Unreleased sections are kept whole. These are matched tolerantly:
      • ## vX.Y.Z (unreleased) in any case or spacing;
      • ## Unreleased, ## [Unreleased] and ## [X.Y.Z] - unreleased;
      • failing safe, any other heading for this version that isn't the dated ## vX.Y.Z (YYYY-MM-DD) the tool writes, such as ## vX.Y.Z (draft) or a bare ## vX.Y.Z;
      • a dated section the tool never wrote lists for (no generated heading).
    • Rerun-safe by construction. When the generated headings can't show where the notes end (the notes use one of them), it writes one end-of-notes line after the notes: NOTES_END, an HTML comment, invisible when rendered. Before returning, it checks that the new section reads its notes back (notesOf) exactly. If they don't, it throws and nothing is written. That is a runtime check, not a lint.
    • Fails closed on ambiguity. It refuses any second section that could be this version's: two unreleased, one unreleased beside a dated one, or two dated (round 2).
    • Never guesses a version (round 2). An unreleased heading counts as versionless only when it has no digit at all. One naming another version, even partly (## v0.6 (unreleased), ## Unreleased (0.6.x), two versions), is left where it is, whole.
    • Conservation check on its own result (round 2, runtime). Every non-blank input line must be in the output (counted), except the fixed intro and the replaced section's heading and generated lists (generated headings, - … (#N) PR lines, the "none" lines and NOTES_END). If one isn't, it throws, naming the lines, and nothing is written. This catches any shape the parser misreads, not only the ones tested.
    • Released (dated) sections go through the same reader as before: the notes above the first generated heading. The difference: a hand-written line after their lists, which main silently dropped, is now refused unless list-shaped (see "The boundary, stated").
  • CHANGELOG.md: adds the Release: an (unreleased) CHANGELOG section survives release.mjs whole, by design (no heading-layout rule) #811 bullet under ### Release tooling in the single ## v0.5.3 (unreleased) section, after main's fold (Changelog: v0.5.3 back to unreleased (one release when everything is done) #850, merged in at 2ce7726). Measured in memory on this tree: changelog 0.5.3 keeps all 151 hand-written lines of that section, on the first run and on a rerun (fold-check.mjs). Main's closing empty ### Guardrail changes stays as part of the notes, so the dated section would show it plus the generated one, with the end-of-notes line between them.
  • test/changelog.test.ts:
    • Changed Release notes v0.5.3: every install-affecting change since v0.5.2, the v0.5.2 NVML correction, notes the release keeps [PROTECTED: owner merges] #809's test. Its first assertion recorded the drop as the fixture's premise (['- a hand-written guardrail note'] dropped). It now asserts nothing is dropped. Its "fixed layout" assertion and its loop over the repo's unreleased sections are unchanged, except that the loop's failure message no longer tells writers to follow a layout. This tightens the test, not weakens it.
    • Adds 5 tests:
      • nine heading spellings, with notes that use every generated heading and the tool's own lines;
      • notes holding the end-of-notes line itself;
      • the refusals, and an unreleased section of another version left alone;
      • a section dated by hand, with no lists, kept whole;
      • the repo's released sections get no end-of-notes line and keep the notes they had.
  • scripts/release.mjs: header comment only.

v0.5.3 cut

Not affected. v0.5.3 was dated by #828 before this merges. Main (71316eb) is merged in.

On #828's CHANGELOG.md (1d3ee82) and on this branch's, regenerating each of v0.5.3, v0.5.2, v0.5.1 and v0.5.0 with this lib is byte-identical to main's lib (measured: parity828.mjs in my scratch dir, library calls only).

Round-1 measurement, kept for the record: unchanged in output, and safer (measured). I ran main's changelog.mjs on main's CHANGELOG.md (with the layout heading) and this PR's changelog.mjs on the same file without it. Both used the same PRs and date, through the library functions, writing nothing. The resulting CHANGELOG.md is byte-identical, and all 3 released versions regenerate byte-identically. Script: parity.mjs in my scratch dir. Output:

v0.5.3 cut, old lib + layout vs new lib without layout: IDENTICAL
old lib, no layout heading: drops 148 of 148 hand-written lines
new lib, no layout heading: drops 0 of 148
released versions: 3 identical, 0 different

"Safer": the same notes can't be lost by a later edit that adds or removes a heading (inferred from the tests below).

How it was verified

  • Fail-without, round 1 (measured by the reviewer, issuecomment-6277852343). Fresh worktree at d5674d0 with this PR's test/changelog.test.ts written in, and main's lib and CHANGELOG.md: 6 of 15 fail (# tests 15 # pass 9 # fail 6). Tests 8, 9, 11 and 12 fail on behaviour; tests 10 and 13 fail because notesOf is a new export. My round-1 figure (7 of 15) was made by swapping the lib in place with git checkout <ref> -- <path>, a method the owner's rules now forbid, so it is withdrawn.
  • Fail-without, round 2 (measured; tests written first and run on the unchanged head 951ca8d before any lib change, no file swapping): 4 of 19 fail:
    not ok 12 - text the step cannot place in a section is never dropped: it refuses and writes nothing   ('Missing expected exception: no space after ##')
    not ok 13 - a dated section's hand-written line after its generated lists is never dropped: the step refuses   ('Missing expected exception.')
    not ok 14 - two dated sections for the version are refused, never merged into one   ('Missing expected exception.')
    not ok 15 - a heading that names another version, even partly, is never taken as this release: it stays as it was   ('## v0.6 (unreleased): not this release's notes')
    
  • With the change (measured, after merging main): changelog, comment-policy-cli, owner-terminal-removed and hotspots (every test that reads CHANGELOG.md or the lib, except release.test.ts): 37/37 pass.
  • Fail-without, round 3 (measured, method b). I committed the tests alone (9b03a56), then ran them in a fresh worktree at that commit, so with the unchanged lib: 1 of 20 fails: not ok 16 - a heading that names another version, even partly, … with ## v0.6 (unreleased): not this release's notes. The boundary test passes there, because it pins current behaviour. With the fix (19f297e): 38/38 pass over changelog, comment-policy-cli, owner-terminal-removed and hotspots. Typecheck is clean and denylist: 1309 files clean (measured). Regenerating v0.5.0 to v0.5.3 on main's CHANGELOG.md is still byte-identical to main's lib (measured).
  • Changed my own round-1 fixture. One test used a custom intro ("Every release, newest first."). The step replaces the intro with the fixed one, so the conservation check now refuses it. The fixture now has the real intro, and new assertions check that a custom intro is refused and the real one isn't.
  • Not run locally: test/release.test.ts and fullsuite.sh. They run release.mjs in temp repos and make throwaway signed tags, and my brief forbids creating tags. CI runs them.
  • npm run typecheck: clean (measured). npm run check:denylist: 1309 files clean (measured, round 3). check:dco origin/main HEAD: all commits signed off (measured).
  • Package size: no packaged files change (measured). npm pack --dry-run --json lists scripts/setup/app-permissions.mjs as the only file under scripts/, and no CHANGELOG.md or test/.

Adversarial pass

Each attack ran against the library in a scratch script (attack.mjs), with a first run and then a rerun that adds a PR. Every result is measured.

Attack Result
CRLF file, notes under ### Guardrail changes 0 lines dropped, comparing lines without their \r (measured). Whether CRs are kept is not compared
Notes are only a bare generated heading 0 dropped
Notes are only - no merged PRs since the previous release (the old reader strips this line) 0 dropped, on the first run and the rerun
Notes contain the end-of-notes line, twice 0 dropped (it reads up to the last one)
A PR titled exactly as the end-of-notes line 0 dropped (a PR line starts - , so it can never equal the marker line)
## Unreleased below an older dated section 0 dropped; moved to the top
A non-version ## Notes inside the notes 0 dropped
Trailing spaces on the heading; no # Changelog head 0 dropped
## v9.9.90 (unreleased) when releasing 9.9.9 not taken; kept whole
Dated and unreleased sections for the same version refused, naming both headings
## vX.Y.Z (draft), bare ## vX.Y.Z, a dated section with no lists first found by this pass, where they would have gone through the old reader. Fixed: all kept whole, and covered by tests

Residual (measured, unchanged from main): a ## v... heading that isn't a version (for example ## very important) inside the notes still starts a new part, as on main. Its lines stay in CHANGELOG.md (0 dropped) but sit after the version's section, so they aren't in the tag notes. Not fixed here: changing the section split for released sections is outside #811.

Adversarial pass, round 2

Measured with the same attack.mjs, extended with the reviewer's shapes.

Attack Result
##v9.9.9 (unreleased), NBSP after ##, ## [9.9.9], ## [9.9.9] - date, ## Next release, notes under the head, a custom intro refused, naming the lines (was: every line dropped)
A hand line after a dated section's lists Corrected in round 3: refused only when it isn't shaped like a list line (a plain line, - a note with no PR number, a non-generated heading, or text under a generated heading). A list-shaped hand line there (- Upgrade first (#77): run doctor, - Extra note, see the PR (#77), - none in this release, a bare generated heading) is replaced silently, as on main (measured by the reviewer, and pinned by a test)
Two dated sections refused (was: second dropped)
## v0.6 (unreleased), ## Unreleased (0.6.x) left whole in place, not adopted (was: adopted)
Fenced ## Unreleased in the notes refused (more than one section)
Fenced ## v9.9.8 (…) in the notes 0 dropped (the "## v…" residual below still applies)
Every round-1 attack unchanged: 0 dropped or refused

Residual, unchanged: a ## v… heading inside the notes (including in a code fence) splits the section. Its lines stay in the file (0 dropped) but sit after the lists.

Adversarial pass, round 3

Measured with attack.mjs and the tests.

Attack Result
## v0.6 (unreleased) (full-width), ## Unreleased (v٠.٦) (Arabic-Indic), ## Unreleased (v²), ## Unreleased (Ⅵ) left in place, not adopted (was: adopted). The rule is now /\p{N}/u: any numeric character
## Unreleased (v9.9.9) (full-width, when releasing 9.9.9) left in place, not adopted. Only an ASCII X.Y.Z names a version exactly, so fail-closed: the 9.9.9 section is written without those notes and they stay where they were, 0 dropped
## Unreleased (next minor) (no numeral) adopted, as designed
List-shaped hand lines after a dated section's lists replaced, as on main; pinned by a test (below)

The boundary, stated (round 3):

  • Replaced. In a dated section the tool wrote lists for, every line after the notes shaped like a generated line is taken as the tool's and replaced, even one written there by hand. That means a generated heading, - text (#N) with or without : … (any text after the colon), - none in this release and - no merged PRs since the previous release. This is what main always did.
  • Refused. Any other line there. The test "the boundary in a dated section's lists …" pins both sides.
  • Proposal, for the owner (not built): refuse when an old list line names a (#N) that the new lists don't list. A hand note like - Upgrade first (#77): run doctor would then be refused instead of replaced. The cost (inferred) is a loud refusal on a rerun whose range no longer holds a PR the first run listed.

What this can never do

  • Drop a non-blank line of CHANGELOG.md other than these:

    • the fixed intro;
    • the replaced section's heading;
    • in a dated section the tool wrote lists for, the list-shaped lines after its notes (the boundary above, as on main).

    withSection checks its own result and throws, so release.mjs changelog writes nothing. That covers unplaceable text, a second dated section, and anything the parser misreads.

  • Take a section whose heading names another version, even partly or in any numerals, as this release's notes.

  • Change a released (dated) section's regenerated output. Measured byte-identical for all four in this repo (v0.5.0 to v0.5.3, on main after Release v0.5.3: changelog and version [PROTECTED: owner merges] #828).

  • Tag, sign, push or publish. This is the changelog step only. The tag step reads the section as before.

Compromised component

  • Agent user / coordinator / dashboard / other instances / hub viewer: none of them runs release.mjs. The owner runs it on a release machine. Nothing here is loaded by the engine (no packaged file changes).
  • Someone with push access, or a PR author editing CHANGELOG.md:
    • They could put text in an unreleased section, and it is now always kept. Before, text under a generated-named heading could vanish silently. Release notes are reviewed in the changelog PR, as before.
    • They could plant the end-of-notes line. That only moves where notes end within that section's text, and it never drops lines (tested).
    • They can't make the step drop the owner's notes. They can make it refuse, by adding a second section for the version, which is fail-closed and loud.
  • A PR title (from GitHub): it can't impersonate the end-of-notes line (shown above).

New dependencies

None.

Stacked on this: the PR for #712 (base issue-811).

…elog step, under any heading

The changelog step kept only what sat above the first heading it also writes ("### Guardrail changes" and the
like), so hand-written notes under such a heading were dropped, and a section with none lost every line (#671).
#809 guarded it by layout: end the notes with an empty generated heading. That is a rule a writer must follow.

Now withSection keeps an unreleased section whole (`## vX.Y.Z (unreleased)` in any case or spacing,
`## Unreleased`, `## [Unreleased]`, and, failing safe, any heading for the version that isn't the dated
`## vX.Y.Z (YYYY-MM-DD)`, or a dated one the tool never wrote lists for): its heading becomes the dated one and
the generated lists go after it.
When the generated headings can't show where the notes end, it writes an end-of-notes line (an HTML comment)
after them, and it checks that the section reads its notes back exactly before returning, so a rerun keeps
them too; otherwise it throws. Two unreleased sections that could both be the version's, or one beside a dated
one, are refused. Released (dated) sections regenerate byte for byte as before.

#809's test asserted the drop as the fixture's premise; it now asserts nothing is dropped.

Test (fails without): test/changelog.test.ts, "an unreleased section, under any spelling of its heading, ...",
"an unreleased section is all hand-written, ...", and three more.

Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
…ges" heading

That heading was #809's layout: the only way to stop the changelog step dropping the notes above and under it.
The step now keeps an unreleased section whole (previous commit, tested), so the heading would only show up as
an empty duplicate next to the generated "Guardrail changes" list. With it removed, the v0.5.3 cut writes the
same CHANGELOG.md as main's step did with it (measured, byte for byte). Adds the #811 bullet.

Test (fails without): test/changelog.test.ts "every section of the repo's CHANGELOG.md keeps its hand-written
text ..." fails on main's changelog.mjs with this CHANGELOG.md (it would drop v0.5.3's notes).

Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
@jayden1711

Copy link
Copy Markdown
Owner Author

Evaluator verdict: CHANGES NEEDED (round 1/2)

Reviewed head 951ca8d against #811. Each claim below is labelled measured, inferred or guess.

The core design is right. The library keeps an unreleased section whole, renames it to the dated heading, appends the lists, and checks the read-back at runtime. Fail-without, typecheck, denylist, CI and the v0.5.3 parity all hold up. However, attacks found three ways the library still loses or misplaces hand-written notes without an error. That conflicts with the issue's outcome ("the release never drops a hand-written note") and with this PR's own "What this can never do".

Acceptance criteria (#811)

Criterion Result Evidence
An (unreleased) section is kept whole and the lists are appended after it met for every heading the splitter recognises measured: 9 heading spellings pass; attack script: tabs, CRLF, NBSP inside the heading, notes made only of a generated heading, and the NOTES_END line planted in the notes (bare, indented, with a trailing tab) all drop 0 hand lines on the first run and on a rerun
Matched tolerantly (case, spacing, missing version) partly measured: ##v0.5.3 (unreleased) (no space), ## + NBSP + v…, ## [9.9.9] and ## [9.9.9] - 2026-12-01 aren't recognised as sections, and every line under them is dropped (required change 1)
Keep #809's test, and add a test that a section using any heading name keeps every line met measured: test/changelog.test.ts 15/15 pass on the head
Remove the layout requirement from docs/releasing.md met (there was none) measured: grep -n -iE "changelog|unreleased|heading" docs/releasing.md on main finds no layout rule. One instruction is left over: test/changelog.test.ts:109's failure message still says "end the hand-written notes with a generated heading such as ### Guardrail changes" (required change 4)
No change for already-released sections met measured: test 13 passes on the head (each released section's notes equal what main's lib kept, and none gets NOTES_END). A rerun on #828's dated v0.5.3 is byte-identical to main's lib

Fail-without (new method: fresh worktree at d5674d0, with the PR's test/changelog.test.ts written in, and main's lib and CHANGELOG.md)

measured: node --import ./dist/test/isolate.js --test dist/test/changelog.test.js gives # tests 15 # pass 9 # fail 6. Tests 8, 9, 11 and 12 fail on behaviour. Test 8's key line:

notes under a heading the tool also writes are kept (#811)
+ [ '- a hand-written guardrail note' ]

Tests 10 and 13 fail on c.notesOf is not a function (a new export). The body's "7 of 15" is also right, but under different conditions: it used this PR's CHANGELOG.md (inferred). With main's CHANGELOG.md, test 7 passes because the layout heading is still there. Note: the body's method (git checkout origin/main -- scripts/lib/changelog.mjs) is the one the owner's review rules now forbid. Please use a fresh worktree next round.

On the head: changelog, comment-policy-cli, owner-terminal-removed and hotspots give 33/33 pass (measured).

Attacks (measured, script: library calls only, writes nothing)

Attack This PR main
CRLF file, notes under ### Guardrail changes 0 hand lines dropped. The # Changelog head is kept twice, because the head regex wants \n drops the guardrail note; same double head
Tabs: ##\tv9.9.9\t(unreleased) 0 dropped n/a
NBSP inside the heading 0 dropped (renamed) 0 dropped (left as is)
NBSP after ##, or ##v9.9.9 (unreleased) with no space, as the first section every line dropped, on the first run and on a rerun same
## [9.9.9], ## [9.9.9] - 2026-12-01, ## Next release, or notes straight under # Changelog every line dropped same
Fenced ```## Unreleased``` in the notes throws (fail closed) n/a
Fenced ```## v9.9.8 (…)``` in the notes splits the section; 0 dropped, but the rest of the notes lands after the lists (the residual the body names) same
Two dated ## v9.9.9 (…) sections the second section's notes are dropped, on the first run and on a rerun same
Dated plus bare ## v9.9.9 throws drops the first section's notes
### Guardrail changes written by hand kept; NOTES_END written; reads back dropped
## v9.9.90 (unreleased), ## Unreleased (9.9.90) when releasing 9.9.9 left alone n/a
## v0.6 (unreleased) or ## Unreleased (0.6.x) when releasing 0.5.3 taken as v0.5.3's notes (no X.Y.Z, so it is read as versionless) n/a
NOTES_END in the notes, twice, indented, or with a trailing tab; an unclosed <!-- 0 dropped; read-back holds n/a
A ## …unreleased… heading inside an older released section every later changelog run throws (fail closed, loud) n/a

Required changes

  1. Fail closed on text the splitter can't place. withSection throws away everything between # Changelog and the first recognised section without an error (measured above). That includes a section the PR says it keeps: ## [9.9.9] (its own versionIn reads 9.9.9) and ##v… (unreleased). This is the same as main, so it isn't a regression (measured). But it makes "can never drop a line of any non-dated section for the version" false, and Release: an (unreleased) CHANGELOG section survives release.mjs whole, by design (no heading-layout rule) #811's outcome is "never drops a hand-written note". Fix it by design, in the library at runtime. The class fix is preferred: check that every non-blank input line, outside the replaced section's generated lists and headings and the fixed intro, appears in the output; if not, throw and write nothing. The narrow fix is to refuse when the stripped head region holds anything but the intro. Add tests for the shapes above.
  2. Two dated sections for the version must be refused. Today the second one's notes are dropped (measured). if (dated.length > 1) throw closes it, and so does the class fix in item 1. Correct the body's ambiguity claim to match.
  3. Don't guess a version. A heading with "unreleased" and a version-like token that isn't X.Y.Z (## v0.6 (unreleased), ## Unreleased (0.6.x)) is treated as versionless, so it becomes this release's notes (measured). The test title promises "which notes are meant is never guessed". Treat only a heading with no version-like token as versionless; refuse the rest (or keep them in place). Add a test.
  4. Remove the leftover layout instruction. test/changelog.test.ts:109's failure message (and the comment at :99) still tell writers to end their notes with a generated heading. That is Release: an (unreleased) CHANGELOG section survives release.mjs whole, by design (no heading-layout rule) #811 item 3. Change only the message text and keep the assertion.
  5. Sequencing with Release v0.5.3: changelog and version [PROTECTED: owner merges] #828 (v0.5.3 release prep). measured: git merge-tree --write-tree 1d3ee82e 951ca8d3 gives CONFLICT (content) in CHANGELOG.md, in the v0.5.3 ### Guardrail changes / ### Release tooling hunk. Release v0.5.3: changelog and version [PROTECTED: owner merges] #828 already dates v0.5.3 and drops the empty layout heading itself. If Release v0.5.3: changelog and version [PROTECTED: owner merges] #828 merges and v0.5.3 is tagged first (recommended), merge main in. Drop this PR's layout-heading removal, which is moot by then (inferred). Move the Release: an (unreleased) CHANGELOG section survives release.mjs whole, by design (no heading-layout rule) #811 bullet into a new ## v0.5.4 (unreleased) section, not into v0.5.3's released notes.

v0.5.3 / #828 interplay (measured, library calls, writes nothing)

Durability rule

Met. The protection is runtime behaviour in scripts/lib/changelog.mjs: withSection throws on ambiguity and on a failed read-back, and release.mjs changelog calls it before writeFileSync (measured from the diff). There is no lint, and the body names the runtime check. One note: the read-back throw is unreachable with generated lists, because a PR line always starts with - (inferred). It is defensive rather than load-bearing, which is fine.

Protection, scope, hygiene

  • Protected paths: scripts/** (engine in main's .worklane/review.yaml:7) (measured). Correctly marked: the title, needs-owner, **Risk class:** protected, and real "What this can never do" and "Compromised component" sections. "Can never do" item 1 is false as written (required changes 1 and 2).
  • Scope: CHANGELOG.md, the lib, the release.mjs comment and the test only (measured). No scope creep.
  • Commits: 2, both Signed-off-by (measured, git log --format=%(trailers:only)). No attribution in commits or body (measured, grep).
  • No packaged file changes (measured: npm pack --dry-run --json lists only scripts/setup/app-permissions.mjs under scripts/, and no CHANGELOG.md, docs or test). No new dependencies (CI new-deps pass). Typecheck clean and denylist: 1308 files clean (measured on the head).
  • Claims in the body are labelled. "Adversarial pass" section present.
  • Real state: the tests are pure library calls plus a read of the repo's CHANGELOG.md, so nothing touches real state (inferred from the test file).

CI (measured, gh pr checks 829, run 39324316284 on 951ca8d)

All required jobs pass: test shards 1-4 on Node 22 and 24, dco, denylist, scan, scope, push-gate, agentshield and new-deps. The real-systemd jobs are skipped, as usual for this path set. test/release.test.ts runs in those shards; I didn't run it locally because it signs throwaway tags.

…place, and never guesses a version

Round 1 review of #829 measured three ways withSection still lost hand-written text without an error:
- text it couldn't place in a section (`##v… (unreleased)` with no space, NBSP after `##`, `## [9.9.9]`,
  `## Next release`, notes straight under `# Changelog`) was thrown away with the old head;
- of two dated sections for the version, the second one's notes were dropped;
- `## v0.6 (unreleased)` and `## Unreleased (0.6.x)` were read as versionless, so they became this release's
  notes.

Now withSection checks its own result: every non-blank input line, except the fixed intro and the replaced
section's heading and generated lists, must be in the output (counted). If one isn't, it throws and nothing is
written. That covers any shape the parser misreads, not only these. It also refuses two dated sections, and an
unreleased heading is versionless only when it has no digit at all; one naming another or an unclear version is
left where it is. The leftover layout advice in the guard test's comment and message is gone.

Test (fails without): test/changelog.test.ts "text the step cannot place in a section is never dropped ...", "a
dated section's hand-written line after its generated lists ...", "two dated sections for the version are
refused ...", "a heading that names another version, even partly ..." (4 of 19 fail on 951ca8d's lib).

Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
CHANGELOG.md: main's v0.5.3 section taken as it is, so this branch's removal of the empty layout heading is
dropped (#828 already dated v0.5.3 without it). The #811 bullet moves to a new `## v0.5.4 (unreleased)` section.

Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
@jayden1711

Copy link
Copy Markdown
Owner Author

Round 2. All four required changes are done, and main (71316eb, #828) is merged in. Head e0c093b.

  1. Text it can't place. withSection now checks its own result, at runtime. Every non-blank input line must be in the output (counted), except the fixed intro and the replaced section's heading and generated lists. Otherwise it throws, names the lines, and writes nothing.
    • Each of your shapes is refused, with a test per shape (measured): ##v…, NBSP after ##, ## [9.9.9], ## [9.9.9] - date, ## Next release, notes under the head.
    • Also refused: a custom intro, and a hand line after a dated section's lists (main dropped that too).
  2. Two dated sections are refused. Tested (measured).
  3. No guessing. An unreleased heading is versionless only with no digit at all. ## v0.6 (unreleased), ## Unreleased (0.6.x), ## Unreleased (9.9 or later) and two versions stay where they are, whole, and are never adopted. Tested (measured).
  4. Layout wording in the test's comment and message is gone. The assertion is unchanged.
  5. Sequencing. The layout-heading removal is dropped (Release v0.5.3: changelog and version [PROTECTED: owner merges] #828 did it). The bullet moved to a new ## v0.5.4 (unreleased). The CHANGELOG diff against main is that section only (measured, git diff origin/main -- CHANGELOG.md).

Fail-without (measured; tests written first and run on the unchanged 951ca8d, with no file swapping): 4 of 19 fail. With the fix, after the merge: 37/37 (changelog, comment-policy-cli, owner-terminal-removed, hotspots). The body now cites your round-1 measurement (6 of 15) instead of my withdrawn one.

Parity (measured, library calls): on main's CHANGELOG.md after #828, regenerating v0.5.0 to v0.5.3 with this lib is byte-identical to main's lib.

Checks (measured): typecheck clean, denylist: 1309 files clean, DCO all signed off.

@jayden1711

Copy link
Copy Markdown
Owner Author

Evaluator verdict: CHANGES NEEDED (round 2/2)

Reviewed head e0c093b (branch commit 31193c5, then a merge of main 71316eb / #828). Each claim below is labelled measured, inferred or guess.

The round-1 changes are in, and they are by design. withSection now checks its own result at runtime and throws instead of writing. Every round-1 shape is refused, two dated sections are refused, and partial versions are no longer adopted. Two attacks still beat it, both small:

  • a hand line shaped like a generated list line, after a dated section's lists, is dropped silently;
  • a version written in non-ASCII digits is still adopted.

Round-1 required changes

# Change Result Evidence
1 Text the splitter can't place is refused met, except item A below measured: round-1 shapes A3, A3c, A3d, A3e, A3f and A11, plus a custom intro and ## Next (v-next) under the intro, all throw the changelog step would drop N line(s) it can't place, so nothing is written. Duplicate hand lines are counted correctly (2 kept plus 1 under the intro throws, naming - same). An indented duplicate under the intro throws
2 Two dated sections are refused met measured: A5, A5b and A5c throw more than one section for v9.9.9
3 No version is guessed met for ASCII digits, not otherwise (item B) measured: ## v0.6 (unreleased), ## Unreleased (0.6.x), ## Unreleased (q4fixes) and ## Unreleased: 2nd batch are left in place, not adopted. ## Unreleased, ## Unreleased (next minor) and ## v-next (unreleased) are adopted, as designed
4 Layout wording removed met measured: the diff changes only the comment and the message of the existing #809 test, and the assertion is unchanged
5 Sequencing with #828 met measured: git merge-tree --write-tree 71316ebd e0c093b5 is clean, and 71316eb is an ancestor of the head. The CHANGELOG diff against main adds only ## v0.5.4 (unreleased) with the #811 bullet; v0.5.3 is untouched

Fail-withouts (method (b): fresh worktrees, tests written in with Write, byte-identical to the head's file per cmp)

  • Against the PR's base, main 71316eb (measured): # tests 19 # pass 9 # fail 10. These fail on behaviour:

    Tests 10, 15 and 17 fail on c.notesOf is not a function, a new export.

  • Against round-1 head 951ca8d, the round-2 delta (measured): # tests 19 # pass 15 # fail 4. All four fail on behaviour:

    • 12: Missing expected exception: no space after ##;
    • 13 and 14: Missing expected exception.;
    • 15: ## v0.6 (unreleased): not this release's notes, with + '### Guardrail changes\n\n- for later' against - ''.

    This matches the builder's "4 of 19".

  • Head (measured): changelog, comment-policy-cli, owner-terminal-removed and hotspots give 37/37 pass.

Round-2 attacks (measured, library calls only, writes nothing)

Attack Result
Duplicate hand lines: 2 in the notes and 1 under the intro refused, naming - same
A dated section with a hand line after its lists shaped like a list line: - Extra note, see the PR (#77), - Upgrade first (#77): run doctor, - none in this release written; the line is dropped (item A)
A dated section with a plain hand line, or a hand ### Fixes, after its lists refused
Lines under the intro that equal generated lines (### Features, - A PR (#5), - none in this release) written. Those lines are gone from where they were, but the same text is in the new lists, so no unique text is lost (named gap)
Whitespace: trailing spaces, tab-indented code in the notes kept byte for byte (0 lost)
Whitespace: an indented copy, under the intro, of a line that is in the notes refused
CRLF file the first run writes, keeping the old CRLF head as a second # Changelog (as on main); the rerun refuses (named gap)
A notes line equal to the intro kept
## v0.6 (unreleased) (full-width), ## Unreleased (v٠.٦) (Arabic-Indic), ## Unreleased (v²) adopted as this release's notes (item B)
Fenced ## Unreleased in the notes refused
Fenced ## v9.9.8 (…) in the notes splits the section, 0 dropped (the residual the body names)
NOTES_END planted, indented, or with a trailing tab; an unclosed <!-- 0 dropped
Regenerating v0.5.0 to v0.5.3 on main's CHANGELOG.md byte-identical to main's lib, for all four
A v0.5.4 cut on the head's CHANGELOG.md written, 0 hand lines lost

Required changes

A. A list-shaped hand line in or after a dated section's lists is dropped without a refusal (measured, above). The conservation check exempts every line in the replaced dated section that matches isListLine, wherever it sits. A hand note such as - Upgrade first (#77): run doctor, added after the lists, vanishes on a rerun. That makes two of the body's claims false:

  • "What this can never do: Drop a non-blank line … other than the fixed intro and the replaced section's heading and generated lists" (the hand line isn't a generated line);
  • the round-2 table row "A hand line after a dated section's lists: refused".

Main dropped such lines too (inferred: main kept only the lines above the first generated heading), so this is not a regression. Minimum (principle 5): correct both claims to state the boundary: inside a dated section's lists region, list-shaped lines are treated as the tool's and replaced. Add a test that pins the boundary. Optional, owner's call (new cost): refuse when an old list line names a (#N) that the new lists don't list, at the price of loud refusals on some reruns.

B. Non-ASCII digits get past the "no digit" rule (measured, above). unreleasedVersion tests /\d/, which is ASCII-only in JavaScript without the u flag. Use /\p{Nd}/u (or \p{N}), so ## v0.6 (unreleased) is left in place like ## v0.6 (unreleased). Add a case to the "names another version, even partly" test.

Named gaps (not blocking)

Rule 10 audit (changed fixture)

Not a weakening (measured from the diff):

Durability rule

Met. The protection is a runtime check of the result inside withSection, before release.mjs changelog writes. It is not a lint, and the body names it (INVARIANT: no hand-written line is dropped).

Protection, scope, hygiene

  • Protection: scripts/** (engine) (measured). Marked protected: the title, needs-owner, risk class, and real "What this can never do" and "Compromised component" sections. One "can never do" item needs the correction in A.
  • Scope: the lib, the release.mjs comment, the test and the CHANGELOG v0.5.4 bullet (measured).
  • Commits: all signed off, including both merges (measured). No attribution in commits, body or reply (measured).
  • Packaging: no packaged files change (inferred: same file set as round 1, where npm pack --dry-run showed none). No new dependencies (CI new-deps pass).
  • Checks: typecheck clean; denylist: 1309 files clean (measured).
  • Body: claims are labelled. "Adversarial pass, round 2" is present and the fail-without is now cited from method (b). The body's checks line still says 1308 files clean, which is stale.

CI (measured, gh pr checks 829, run 39331938778 on e0c093b)

All pass: test shards 1-4 on Node 22 and 24 (including release.test.ts), dco, denylist, scan, scope, push-gate, agentshield, new-deps and desktop. The real-systemd jobs are skipped.

Merge order around v0.5.3

v0.5.3 isn't tagged yet (measured: git ls-remote --tags origin v0.5.3 is empty).

  1. Cut v0.5.3 from main as it is (71316eb).
  2. Merge Release: an (unreleased) CHANGELOG section survives the changelog step whole, by design [PROTECTED: owner merges] #829, then Release: a GitHub-button revert is read from its body, and the tag step refuses an (unreleased) section [PROTECTED: owner merges] #831 right after it, before the v0.5.4 cut.

If this merges before the v0.5.3 tag, the tag would carry code whose notes say v0.5.4 (inferred).

jayden1711 added a commit that referenced this pull request Oct 19, 2026
…ding (#783)

Until #829 lands, the release lib keeps an unreleased section's hand-written
notes only above a generated heading, as #834 and #839 do.

Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
…-lists boundary

Round 2 review of #829: `## v0.6 (unreleased)`, `## Unreleased (v٠.٦)` and `## Unreleased (v²)` were adopted as
this release's notes, because the "no digit" rule is ASCII only. Adds those cases (they fail until the next
commit), and a test pinning the boundary in a dated section's lists: list-shaped lines there are replaced, as
on main; any other line is refused.

Test (fails without): this commit holds the tests only; the next commit is the fix.

Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
…sts boundary is pinned

Round 2 review of #829 measured two gaps:
- The "no digit" rule for a versionless unreleased heading tested /\d/, which is ASCII only, so `## v0.6
  (unreleased)`, `## Unreleased (v٠.٦)` and `## Unreleased (v²)` were adopted as this release's notes. It now
  tests /\p{N}/u: any numeric character, so they stay where they are, whole.
- In a dated section the tool wrote lists for, a hand line shaped like a list line after the notes (`- Upgrade
  first (#77): run doctor`, `- none in this release`) is replaced silently, as on main. Not changed here: the
  doc comment now states the boundary, and a test pins it (list-shaped lines there are replaced; any other line
  is refused). The stricter option is the owner's call, proposed in the PR body.

Test (fails without): test/changelog.test.ts "a heading that names another version, even partly, ..." (1 of 20
fails at 9b03a56, the tests-only commit, in a fresh worktree; the boundary test pins current behaviour).

Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
@jayden1711

Copy link
Copy Markdown
Owner Author

Round 3. Head 19f297e.

B. Any numerals. unreleasedVersion now tests /\p{N}/u. ## v0.6 (unreleased), ## Unreleased (v٠.٦), ## Unreleased (v²) and ## Unreleased (Ⅵ) are left in place, whole, and not adopted. They are in the "names another version, even partly" test (measured).

A. The boundary. Not changed, as briefed. The doc comment states it, and a new test pins both sides (measured):

  • Replaced: - Upgrade first (#77): run doctor, - Extra note, see the PR (#77), - none in this release, and a generated heading with a PR line, or bare, after a dated section's lists. As on main.
  • Refused: a plain line, - a note with no PR number, a generated heading with plain text under it, and ### Upgrade notes.

The body's two false claims are corrected. The stricter option (refuse when an old list line names a (#N) the new lists lack) is written up as a proposal for the owner.

Fail-without (measured, method b). Tests committed alone (9b03a56) and run in a fresh worktree at that commit: 1 of 20 fails (test 16, ## v0.6 (unreleased): not this release's notes). The boundary test passes there, because it pins current behaviour. With the fix, 38/38 pass. Typecheck is clean and denylist: 1309 files clean. v0.5.0 to v0.5.3 regenerate byte-identically to main's lib.

#831 has this merged in (a merge, not a rebase): head 63b9701, 44/44 pass.

jayden1711 added a commit that referenced this pull request Oct 19, 2026
…s reads and #846

v0.5.3 is released (main 71316eb), so the entry goes in a new v0.5.4 (unreleased) section above it, closed by an
empty "### Guardrail changes" heading, which the release lib needs until #829 lands (test/changelog.test.ts).
main's v0.5.3 text is unchanged (git diff origin/main -- CHANGELOG.md shows additions above it only).

Test (fails without): none; a CHANGELOG change only (test/changelog.test.ts passes).

Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
@jayden1711

Copy link
Copy Markdown
Owner Author

Evaluator verdict: PASS

Round 3, head 19f297e: 9b03a56 (tests only), then 19f297e (the fix). Each claim below is labelled measured, inferred or guess.

Both round-2 items are resolved. B is fixed in code. A is resolved as the decision guide allows: the claims are corrected, the boundary is pinned by a test, and the stricter option goes to the owner as a proposal.

Round-2 required changes

Item Result Evidence
B: non-ASCII numerals met measured, library calls on the head. Every heading below is left whole in place and never adopted when releasing 9.9.9: ## v0.6 (unreleased), (v٠.٦), Devanagari (v०.६), math-bold astral (v𝟎.𝟔) and (v9.9.9) (\p{Nd}); superscript (v²), subscript (v₂), (½), (①) and (⑴) (\p{No}); (Ⅵ), (ⅵ), (〇) and astral (𐍁) (\p{Nl}). These are still adopted, as designed: ## Unreleased, ## Unreleased (next minor) and ## Unreleased (9.9.9). A non-unreleased ## v9.9.9 heading, dated or not, stays in place, and tagNotes refuses it with "no v9.9.9 section"
A: list-shaped lines after a dated section's lists met, by correcting the claims (principle 5) measured: the new test "the boundary in a dated section's lists …" matches the behaviour I attacked. These lines are replaced: - Upgrade first (#77): run doctor, - Extra note, see the PR (#77), - none in this release, ### Fixes plus a PR line, and ### Security and guardrails. These are refused: a plain line, - a note with no PR number, ### Fixes plus a plain line, and ### Upgrade notes. My round-2 attacks C3, C3b and C3d give the same results; C3c is refused. The body's "What this can never do" now lists this boundary, the round-2 table row is corrected, and the number-matching refusal is a proposal for the owner

Fail-without (method (b))

I made a fresh worktree at 9b03a56's parent, e0c093b, and wrote in the head's test/changelog.test.ts (byte-identical per cmp). The result is # tests 20 # pass 19 # fail 1 (measured):

not ok 16 - a heading that names another version, even partly, is never taken as this release: it stays as it was
    ## v0.6 (unreleased): not this release's notes

This matches the builder's "1 of 20". The boundary test passes on the unchanged lib, as expected: it pins behaviour that was already there. 9b03a56 changes only the test file, and 19f297e changes only the lib (measured, git diff --stat).

On the head (measured): changelog, hotspots and comment-policy-cli give 30/30 pass. owner-terminal-removed (which spawns stand-in host tools) ran in CI only. Typecheck is clean, and the denylist reports 1309 files clean.

Other round-2 attacks, rerun on the head (measured)

  • Still refused: unplaceable text, a custom intro, duplicate hand lines and an indented duplicate.
  • Still kept or tolerated: lines that equal newly generated lines, and whitespace-only lines are kept. A CRLF rerun refuses.
  • Regeneration: v0.5.0 to v0.5.3 regenerate identically with the round-2 lib, which was measured identical to main's lib in round 2. So the head is identical to main for released sections; that is measured in two steps. A v0.5.4 cut on the head's CHANGELOG.md loses 0 hand lines.

Non-blocking notes on the body (text only, no re-review needed)

  • "Released (dated) sections" bullet: it still says a hand-written line after the lists "is now refused". Add "unless it is list-shaped", to match the boundary stated below it.
  • The boundary text says "- text (#N) with or without : files". isListLine accepts any text after the colon (: run doctor), and the test pins that. "with or without : …" would be exact.
  • The owner proposal's example cost, "a PR renamed between runs", wouldn't refuse under a number match. A renamed PR keeps its #N (inferred). The real cost is a PR that drops out of the range.

Durability rule, protection and hygiene

  • Durability: met. The protection is the runtime conservation check and version rule inside withSection. There is no lint.
  • Protection: scripts/** (engine) (measured). Marked protected: the title, needs-owner, risk class, and real "What this can never do" and "Compromised component" sections. The "can never do" items hold against the attacks above.
  • Commits: 9b03a56 and 19f297e are both signed off, with no attribution (measured).
  • Packaging: no packaged files change (inferred: only scripts/lib/changelog.mjs and the test changed this round).
  • Merges: git merge-tree --write-tree origin/main 63b97011 (the stack head, which contains this head) is clean against main 71316eb (measured).

CI (measured, run 39336194400 on 19f297e)

All pass: test shards 1-4 on Node 22 and 24, dco, denylist, scan, scope, push-gate, agentshield, new-deps and desktop. The real-systemd jobs are skipped.

Merge order around v0.5.3

v0.5.3 is still untagged (measured: git ls-remote --tags origin v0.5.3 is empty).

  1. Cut v0.5.3 from main 71316eb.
  2. Merge this PR.
  3. Merge Release: a GitHub-button revert is read from its body, and the tag step refuses an (unreleased) section [PROTECTED: owner merges] #831 right after it, before the v0.5.4 cut. Until Release: a GitHub-button revert is read from its body, and the tag step refuses an (unreleased) section [PROTECTED: owner merges] #831 lands, main's tag step reads notes with sectionFor.

… issue-811

Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>

# Conflicts:
#	CHANGELOG.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Release: an (unreleased) CHANGELOG section survives release.mjs whole, by design (no heading-layout rule)

1 participant