Surface report regression evidence in public docs

[luoyuctl]

Background

PRs #203, #204, and #205 added a connected report/CI evidence batch after v0.4.6: local baseline comparison, incident timeline evidence, and conservative tool authority categories.

Evidence

• Add overview baseline comparison #203 added baseline_comparison and local CI baseline comparison fields.
• [Impl] Add incident timeline evidence to TUI and reports #204 added incident_timelines to overview JSON/Markdown/HTML and TUI evidence surfaces.
• Classify tool authority categories #205 added summary.tool_authority, surfaces.authority_categories, and authority-aware baseline comparison behavior.
• Product post-merge validation on master at f354b39 confirmed these fields coexist in demo JSON output.
• Duplicate search on 2026-05-17 found no open public-docs task for packaging this combined report evidence batch.
• PR Add growth assets and npm launch docs #156 is still the active blocked Growth PR touching README/site surfaces, so parallel public-docs edits need an explicit owner decision or sequencing.

User value

Users evaluating agenttrace should quickly understand that reports can now explain regressions with baseline deltas, incident timelines, and tool authority evidence instead of only generic health scores.

Adoption rationale

These features make agenttrace easier to evaluate for CI review and post-run debugging. Public docs should make the evidence model legible without promising hosted tracing, security enforcement, or package publication.

Suggested scope

• Add concise public-doc wording for the combined evidence batch after Add growth assets and npm launch docs #156 is resolved or explicitly split.
• Candidate surfaces: README feature table, site workflow/report section, and release notes draft for the next authorized release train.
• Mention baseline comparison, incident timelines, and conservative tool authority categories as local report evidence.
• Keep npm/package publication state unchanged and cautious.

Non-goals

• Do not publish npm, Homebrew, GitHub releases, tags, or external announcements from this issue.
• Do not claim security certification, policy enforcement, or hosted telemetry.
• Do not create a parallel PR that conflicts with Add growth assets and npm launch docs #156 unless a maintainer explicitly authorizes the split.

Acceptance criteria

• Public docs explain the post-v0.4.6 report evidence batch in concise user-facing language.
• Docs remain local-first and do not introduce hosted-service or security-enforcement claims.
• npm remains described as prepared/unpublished unless Decide npm package publication path before launch docs go live #174 records a different completed package state.
• If Add growth assets and npm launch docs #156 is still open, the owner either folds this scope into Add growth assets and npm launch docs #156 with authorization or records that this waits until after Add growth assets and npm launch docs #156 is resolved.
• Docs/release-surface/page checks pass for any changed public surfaces.

Suggested lane

lane/growth

Risk

Medium. Overstating authority categories could make the feature sound like security enforcement. Keep wording conservative and evidence-oriented.

Source

Product post-merge review of #203, #204, #205 and Discussion #2 feedback.

[luoyuctl]

Event: Evidence
Actor: Product
Scope: Issue #206
State change: no label change; keep lane/growth + status/needs-human
Evidence: Discussion #2 now has a current-reader update linking the feedback thread to the merged master work from #203, #204, and #205: #2 (comment)
Next owner: Human / Growth

Decision: Keep #206 as the public-doc packaging follow-up for the combined evidence model. This should not spawn a parallel README/site PR while #156 remains open unless a maintainer explicitly authorizes the split or decides to fold this scope into #156.

Acceptance check reminder: public docs should describe baseline comparison, incident timeline evidence, and conservative tool authority categories as local report evidence, without claiming hosted tracing, security enforcement, npm publication, release execution, or external posting.

[luoyuctl]

Event: Evidence / Acceptance refinement
Actor: Product
Scope: Issue #206 / public-doc packaging
State change: none; keep lane/growth,status/needs-human
Evidence: PR #156 review found a related public-copy trust risk: the proposed README first-screen proof adds one exact real-run aggregate, while the same README still contains a different exact captured-run table. Live local overview numbers can also drift as new sessions are written.
Next owner: Human / Growth

Decision support: when #206 is implemented, public docs should package the report evidence model without relying on volatile exact live-run totals unless the snapshot source is clearly identified and kept internally consistent. Prefer stable capability wording for baseline comparison, incident timelines, failure families, and conservative tool authority categories; if exact numbers are used, keep them tied to one captured asset/report snapshot.

Acceptance check addition: README/site/release-note copy should not present conflicting exact real-run aggregates in adjacent sections, and should not imply hosted telemetry or uploaded logs.

[luoyuctl]

Event: Evidence / Surface audit
Actor: Product
Scope: Issue #206 / site and demo-report surfaces
State change: none; keep lane/growth,status/needs-human
Evidence: Rechecked current master f354b39. docs/ci-integration.md already mentions baseline_comparison, deterministic regression fields, new tool authority categories, and new high-authority tool use. However, public site surfaces do not yet make the combined evidence model legible:

• site/index.html still describes broad slow-run/regression diagnosis but does not name baseline comparison, incident timelines, or tool authority categories.
• site/demo-report.html contains older report sections such as metrics, recent sessions, and recent anomalies, but no clear incident timeline, baseline comparison, or tool authority category explanation.
• scripts/ci/check-pages-artifact.sh site can pass while this content gap remains, because the check covers page/artifact integrity rather than whether the public example demonstrates new report evidence.
  Next owner: Human / Growth

Decision support: keep #206 as the follow-up for packaging the #203/#204/#205 evidence model. Candidate implementation should update public site/report example copy only after #156 is resolved or explicitly split, and should avoid volatile exact live-run totals unless tied to one captured report snapshot.

Acceptance check refinement: public site/demo-report surfaces should let a first-time evaluator understand baseline comparison, incident timeline evidence, and conservative tool authority categories without implying hosted tracing, security enforcement, or uploaded logs.

[luoyuctl]

Event: Handoff linkage
Actor: Product
Scope: Issue #206 / Issue #207
State change: none
Evidence: Created #207 to cover the Quality side of the static site/demo-report.html gap: generated demo reports expose current report evidence, while the committed static demo report can pass Pages artifact checks without demonstrating those semantics.
Next owner: Human / Growth for #206; Quality for #207

Decision: keep #206 focused on public-doc packaging and wording, while #207 owns the narrow CI/static artifact guard. This avoids turning #206 into a mixed Growth + Quality implementation task.

[luoyuctl]

Event: Evidence
Actor: Product
Scope: #206 public-doc packaging boundary
State change: none; keeping status/needs-human.
Evidence: #208 is merged on master as 1c13313 and post-merge validation passed for Pages artifact, release surfaces, docs command smoke, build, and report semantics. That closes the CI/static-site guard gap for stale demo-report semantics.
Product read: #206 still needs a maintainer/product packaging decision for how public docs should present the newer report evidence without creating volatile or conflicting exact-count claims, especially while #156 remains a draft launch-doc path.

[luoyuctl]

Event: Evidence / Scope refinement
Actor: Product
Scope: Issue #206 after report overview follow-ups
State change: none; keep lane/growth,status/needs-human
Evidence: Since the original #206 evidence batch, additional post-v0.4.6 report-surface work has merged and passed Product validation: HTML overview Tool authority (#210), Markdown overview Tool authority (#212), text overview Incident timeline + Tool authority cues (#214), UTF-8-safe text truncation (#217), readable Tool authority wrapping (#219), and readable Incident timeline row bounds (#221). Current demo output exposes Incident timeline and Tool authority in generated reports; baseline comparison remains present when a baseline input is supplied.
Next owner: Human / Growth

Decision support: #206 should remain the single public-doc packaging follow-up for the report evidence model rather than spawning a parallel docs PR while #156 is unresolved. Candidate public wording can now describe the durable capability shape across JSON/Markdown/HTML/text: local baseline comparison when supplied, incident timeline evidence, conservative tool authority categories, and terminal-readable overview rows. Keep wording local-first and avoid hosted tracing, security enforcement, uploaded logs, package publication, release execution, or volatile exact live-run aggregates.