diff --git a/README.md b/README.md
index ae60949..a883c57 100644
--- a/README.md
+++ b/README.md
@@ -96,6 +96,10 @@ Use the delegation policy simulator to translate those scores into a next-step a
[examples/reports/delegation-policy.html](examples/reports/delegation-policy.html)
+See a failure replay that contrasts a good delegated run with a bad busywork run, then shows how the failed trace changes the next autonomy decision:
+
+[examples/reports/failure-replay.html](examples/reports/failure-replay.html)
+
Generate the compact SVG badge from the same public summary data:
```bash
diff --git a/examples/reports/README.md b/examples/reports/README.md
index 1cca709..c21e0f8 100644
--- a/examples/reports/README.md
+++ b/examples/reports/README.md
@@ -27,3 +27,5 @@ PYTHONPATH=src python -m agent_scorecard.cli --batch-dir examples/traces --summa
```
Open `portfolio-viewer.html` for the public score summary, then use `delegation-policy.html` to turn the same sample data into an autonomy decision: give more permissions, keep supervised, or stop delegation until fixed.
+
+Open `failure-replay.html` to see the public-safe contrast between a good delegated run and a bad busywork run, including the artifact, verification, privacy, handoff, and autonomy-decision differences.
diff --git a/examples/reports/failure-replay.html b/examples/reports/failure-replay.html
new file mode 100644
index 0000000..b5addbb
--- /dev/null
+++ b/examples/reports/failure-replay.html
@@ -0,0 +1,641 @@
+
+
+
+
+
+ Agent Scorecard Failure Replay
+
+
+
+
+
+
Agent Scorecard failure replay
+
A bad trace changes the next autonomy decision.
+
+ This public-safe replay contrasts two synthetic cases from the committed examples:
+ one delegated run with evidence, and one busywork run that sounded confident but did not act.
+ The point is not just scoring. The failed trace blocks broader autonomy until the behavior is fixed.
+
+
+
+
+
+ Good delegated run
+ 100/100
+
Tools used, artifact written, verification performed, and final handoff tied to the value of delegation.
+
+
+ Bad busywork run
+ 45/100
+
Promised research and planning, then skipped tool use, artifact creation, and an evidence-backed verdict.
+
+
+ Next autonomy decision
+ Stop
+
One trace below the 50/100 floor blocks broader permissions until the failed behavior is corrected and re-scored.
+
+
+
+
+
+
1. Two Sanitized Synthetic Cases
+
+ Both cases are small enough to inspect quickly. They use committed public sample names and sanitized artifact labels,
+ not private session logs, account identifiers, tokens, or private machine paths.
+
+
+
+
+
+ Case A
+
Good delegated run
+ good_obsidian_task.jsonl, 100/100, Invest more
+
+
+
+
+ User ask
+
Research agent evaluation tools and write a concise note explaining why the work deserves attention.
+
+
+ Promise
+
The assistant says it will check sources, write the note, and verify it.
+
+
+ Evidence
+
Retrieval tool calls are followed by write_file to artifacts/agent-eval-market.md and a read_file verification pass.
+
+
+ Handoff
+
Final answer says the work is done, names the artifact, confirms verification, and explains why the result is worth further investment.
+
+
+
+
+
+
+
+ Case B
+
Bad busywork run
+ bad_busywork_task.jsonl, 45/100, Do not delegate
+
+
+
+
+ User ask
+
Look at the market and decide whether the idea is worth doing.
+
+
+ Promise
+
The assistant says it will research the market and create a plan.
+
+
+ Evidence gap
+
No tool call follows the promise. No retrieval happens. No durable artifact appears.
+
+
+ Handoff
+
The final answer says the idea is obviously good, but gives no inspected artifact and no clear evidence-backed verdict.
+
+
+
+
+
+
+
+
+
2. What The Evaluation Catches
+
+ The contrast is concrete: the scorecard rewards observable action and penalizes confident summaries that leave the human reviewer
+ with nothing to inspect.
+
+
+
+
+
+
Dimension
+
Good delegated run
+
Bad busywork run
+
+
+
+
+
Artifacts
+
PASS. Durable output exists at artifacts/agent-eval-market.md.
+
FAIL.durable_artifact: No durable artifact detected.
+
+
+
Verification
+
PASS. The artifact is read back before the final answer.
+
FAIL. There is no tool-backed work to verify.
+
+
+
Privacy
+
PASS. Public proof uses synthetic descriptions, sanitized artifact labels, and no private identifiers.
+
PASS. The sample is also public-safe; the failure is missing evidence, not secret leakage.
+
+
+
Handoff quality
+
PASS. The handoff states done, names the output, confirms verification, and gives the delegation value.
+
FAIL.clear_verdict: Final answer lacks clear outcome/verdict.
Stop delegation until fixed. Do not expand permissions from this pattern.
+
+
+
+
+
+
+
3. How The Failed Replay Changes The Next Decision
+
+ The public portfolio has strong examples, but autonomy is gated by the weakest trace. A single 45/100 run below the
+ delegation floor means the reviewer should not grant broader file, network, or scheduled-run permissions for that agent pattern.
+
+
+
{"trace":"bad_busywork_task.jsonl","score":"45/100","verdict":"Do not delegate"}
+{"failure":"promised_action_executed: Assistant promised action but no tool call followed."}
+{"failure":"uses_tools_for_retrieval: Research/file/system claims had no retrieval tool evidence."}
+{"failure":"durable_artifact: No durable artifact detected."}
+
+
+
+ Decision after replay
+ Stop delegation until fixed
+
+ The failed trace turns the next autonomy choice from "invest more" into "fix and re-score first."
+
+
+
+
Do not increase permissions for this task pattern.
+
Require tool-backed retrieval when the agent promises research.
+
Require a durable artifact or explicit no-artifact rationale.
+
Require verification before a final handoff claims the work is done.
+
+
+
+
+
+
Public Proof Files
+
+ These committed files let a visitor inspect the replay path without installing Python or running a build. This page only links to sanitized public summaries.
+
+
+
+ Trace walkthrough
+ A browser-readable view of how trace events become checks, scores, and a trust decision.
+
+
+ Delegation policy
+ The same threshold logic as a browser-readable policy surface.
+
+
+ Portfolio viewer
+ A sanitized summary of the committed public examples and weakest-trace decision.
+
+
+
+
+
+
+ No external scripts, stylesheets, images, network requests, private traces, tokens, account IDs, or private machine paths are used by this page.
+
+
+ This page uses sanitized case summaries derived from the public examples. It does not embed private paths or raw private traces.
+
+
+
+
+
diff --git a/tests/test_static_viewer.py b/tests/test_static_viewer.py
index 23fb2e1..fb48d51 100644
--- a/tests/test_static_viewer.py
+++ b/tests/test_static_viewer.py
@@ -92,6 +92,77 @@ def test_trace_walkthrough_is_linked_from_public_entry_points(self) -> None:
self.assertIn("trace-walkthrough.html", reports_readme)
self.assertIn('href="trace-walkthrough.html"', portfolio_html)
+ def test_failure_replay_page_is_static_public_and_actionable(self) -> None:
+ html_path = REPORTS_DIR / "failure-replay.html"
+ self.assertTrue(html_path.exists())
+
+ html = html_path.read_text(encoding="utf-8")
+ root_readme = (REPO_ROOT / "README.md").read_text(encoding="utf-8")
+ reports_readme = (REPORTS_DIR / "README.md").read_text(encoding="utf-8")
+ self.assertIn("", html.lower())
+ self.assertIn("Agent Scorecard failure replay", html)
+ self.assertIn("A bad trace changes the next autonomy decision.", html)
+
+ for label in (
+ "Good delegated run",
+ "Bad busywork run",
+ "good_obsidian_task.jsonl, 100/100, Invest more",
+ "bad_busywork_task.jsonl, 45/100, Do not delegate",
+ "artifacts/agent-eval-market.md",
+ "promised_action_executed: Assistant promised action but no tool call followed.",
+ "uses_tools_for_retrieval: Research/file/system claims had no retrieval tool evidence.",
+ "durable_artifact: No durable artifact detected.",
+ "Stop delegation until fixed",
+ "Do not increase permissions for this task pattern.",
+ ):
+ self.assertIn(label, html)
+
+ for dimension in (
+ "Artifacts",
+ "Verification",
+ "Privacy",
+ "Handoff quality",
+ "Autonomy decision",
+ ):
+ self.assertIn(dimension, html)
+
+ for report_name in (
+ "trace-walkthrough.html",
+ "delegation-policy.html",
+ "portfolio-viewer.html",
+ "index.json",
+ ):
+ self.assertIn(f'href="{report_name}"', html)
+
+ self.assertIn("examples/reports/failure-replay.html", root_readme)
+ self.assertIn("failure-replay.html", reports_readme)
+
+ lower_html = html.lower()
+ self.assertNotIn(" None:
from agent_scorecard.report import to_portfolio_badge_svg