Title
internal/sanitize: control-char stripping and Markdown escaping
Summary
Implement the text-hygiene primitives applied to all untrusted source
content: control-character stripping (terminal-escape defense), UTF-8
repair, length capping, single-line flattening, and Markdown escaping for
table cells and content lines (DESIGN.md §12.1 B6, §12.2).
Context
History lines, commit subjects, and agent session titles are untrusted bytes
(invariant I5). If rendered raw they can smuggle ANSI escape sequences into
terminals or break/forge Markdown structure in reports. Aggregation (18) and
renderers (20/21) call these helpers; parsers use the UTF-8/caps primitives.
Scope
internal/sanitize/sanitize.go + tests. Stdlib only.
Detailed Requirements
Clean(s string) string:
- Repair invalid UTF-8 (
strings.ToValidUTF8(s, "�")).
- Remove all C0 controls except
\n and \t; remove \r (normalize
CRLF→LF first); remove DEL (0x7F) and C1 controls (U+0080–U+009F).
- Remove Unicode line/paragraph separators (U+2028, U+2029) and BOM.
- Leaves all other printable Unicode intact (Japanese text must pass
through unchanged — test with CJK + emoji).
Line(s string, max int) string — Clean, then replace \n/\t runs
with a single space, trim, and cap to max runes appending … when
truncated (max ≤ 0 = no cap). Truncation must not split a rune.
Cap(s string, max int) string — rune-safe cap with …, preserving
newlines (used for Body/narrative).
EscapeCell(s string) string — for Markdown table cells: apply Line
semantics (no cap), then escape | as \|, backtick as \`, and
HTML-escape < and & (<, &).
EscapeText(s string) string — for non-table content lines: Clean,
HTML-escape < and &, and neutralize a leading #, -, *, >, or
digit-dot list marker by prefixing \ (prevents structure forgery when a
subject starts a line).
- Idempotence contract (documented in doc comments and asserted in tests
over the corpus): Clean, Line, and Cap are idempotent
(f(f(x)) == f(x)). EscapeCell and EscapeText are deliberately NOT
idempotent (escaping & twice yields &amp;): they are
single-application functions applied exactly once, at render time, by the
renderer — never by parsers or aggregation. Their doc comments must state
this.
Acceptance Criteria
Validation
go test -race -cover ./internal/sanitize/ output in PR. Add one Go fuzz
target FuzzClean (seeded with the corpus) run for ≥ 30s locally:
output must always satisfy the "no forbidden bytes + valid UTF-8" property.
Dependencies
Non-goals
Redaction (16/17 — secrets are a separate pass), zsh unmetafy (09 — that is
format decoding, not sanitizing), full HTML sanitization (reports are
Markdown; only </& neutralization is required).
Design References
docs/DESIGN.md §12.1 (B6), §12.2 (terminal escape / markdown injection
rows), §6.2 (Event field hygiene)
Source of truth: docs/issues/06-sanitize-package.md (PR #1, branch docs/v1-design). If this issue and the repo docs disagree, the docs win. Execution order and dependencies: docs/ISSUE_PLAN.md (this is issue 06 of 33).
Title
internal/sanitize: control-char stripping and Markdown escaping
Summary
Implement the text-hygiene primitives applied to all untrusted source
content: control-character stripping (terminal-escape defense), UTF-8
repair, length capping, single-line flattening, and Markdown escaping for
table cells and content lines (DESIGN.md §12.1 B6, §12.2).
Context
History lines, commit subjects, and agent session titles are untrusted bytes
(invariant I5). If rendered raw they can smuggle ANSI escape sequences into
terminals or break/forge Markdown structure in reports. Aggregation (18) and
renderers (20/21) call these helpers; parsers use the UTF-8/caps primitives.
Scope
internal/sanitize/sanitize.go+ tests. Stdlib only.Detailed Requirements
Clean(s string) string:strings.ToValidUTF8(s, "�")).\nand\t; remove\r(normalizeCRLF→LF first); remove DEL (0x7F) and C1 controls (U+0080–U+009F).
through unchanged — test with CJK + emoji).
Line(s string, max int) string—Clean, then replace\n/\trunswith a single space, trim, and cap to
maxrunes appending…whentruncated (
max ≤ 0= no cap). Truncation must not split a rune.Cap(s string, max int) string— rune-safe cap with…, preservingnewlines (used for
Body/narrative).EscapeCell(s string) string— for Markdown table cells: applyLinesemantics (no cap), then escape
|as\|, backtick as\`, andHTML-escape
<and&(<,&).EscapeText(s string) string— for non-table content lines:Clean,HTML-escape
<and&, and neutralize a leading#,-,*,>, ordigit-dot list marker by prefixing
\(prevents structure forgery when asubject starts a line).
over the corpus):
Clean,Line, andCapare idempotent(
f(f(x)) == f(x)).EscapeCellandEscapeTextare deliberately NOTidempotent (escaping
&twice yields&amp;): they aresingle-application functions applied exactly once, at render time, by the
renderer — never by parsers or aggregation. Their doc comments must state
this.
Acceptance Criteria
\x1b]0;evil\x07,\x1b[31mred\x1b[0m,\x9b31m, raw\x07come out with all escapebytes removed (assert no byte < 0x20 except
\n/\tremains, no0x7F, no C1).
Clean.invalid UTF-8 (fuzz-style table over Japanese strings with varying
caps).
EscapeCell("a|bc<d&e")` produces the exact documented escapes; arendered table row containing it still parses as one row (golden
string assert).
EscapeText("# not a heading")andEscapeText("- not a bullet")neutralize the marker; mid-line
#/-untouched.Clean/Line/Capover the whole corpus;for
EscapeCell/EscapeTexta test documents the non-idempotence(double application visibly differs) so callers cannot miss it.
Validation
go test -race -cover ./internal/sanitize/output in PR. Add one Go fuzztarget
FuzzClean(seeded with the corpus) run for ≥ 30s locally:output must always satisfy the "no forbidden bytes + valid UTF-8" property.
Dependencies
Non-goals
Redaction (16/17 — secrets are a separate pass), zsh unmetafy (09 — that is
format decoding, not sanitizing), full HTML sanitization (reports are
Markdown; only
</&neutralization is required).Design References
docs/DESIGN.md§12.1 (B6), §12.2 (terminal escape / markdown injectionrows), §6.2 (Event field hygiene)
Source of truth:
docs/issues/06-sanitize-package.md(PR #1, branchdocs/v1-design). If this issue and the repo docs disagree, the docs win. Execution order and dependencies:docs/ISSUE_PLAN.md(this is issue 06 of 33).