You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 847dac3
Browse filesBrowse the repository at this point in the historyBrowse files
description: Audit a repository for autonomous-agent harness readiness across cold-start docs, rules, lint, hooks, tests, PR review automation, repo skills, and garbage-collection cadence. Use for "harness audit", "agent-ready repo", "harness readiness", "make this repo agent-friendly", "harness gaps", or surgical fixes for top harness gaps.
3
+
description: Audit a repository for autonomous-agent harness readiness and Symphony-style unattended ticket execution readiness across cold-start docs, rules, lint, hooks, tests, PR automation, repo skills, garbage-collection cadence, workflow contracts, evidence, observability, and smoke-ticket evals. Use for "harness audit", "agent-ready repo", "harness readiness", "make this repo agent-friendly", "Symphony readiness", "prepare this repo for Symphony", "ticket-level agent automation", or surgical fixes for top harness gaps.
4
4
---
5
5
6
6
# Harness Audit
7
7
8
-
Audit a codebase against an 8-artifact framework for how well-equipped it is for autonomous AI agents to do end-to-end implementation work. Optionally apply surgical fixes for the highest-leverage gaps.
8
+
Audit a codebase against a baseline 8-artifact framework for how well-equipped it is for autonomous AI agents to do end-to-end implementation work. When asked for Symphony readiness, add the unattended ticket-execution overlay: workflow contract, disposable workspace bootstrap, validation/evidence, ticket lifecycle, observability, safety, and smoke-ticket eval. Optionally apply surgical fixes for the highest-leverage gaps.
9
9
10
10
Framework provenance: Ryan Lopopolo's "Harness Engineering" talk (OpenAI, AI Engineer London 2025). Core thesis: **code is cheap; scarce resources are human time, attention, and model context window.** Codebases that surface the right context to agents at the right time produce more shippable work per human hour.
11
11
@@ -14,9 +14,49 @@ Framework provenance: Ryan Lopopolo's "Harness Engineering" talk (OpenAI, AI Eng
14
14
-**Repo architecture** — structural design and repo organization. Use that for "how should this repo be laid out?"
15
15
-**Adversarial review** — hunts real bugs in working or near-complete implementations. Use that for "is this code correct?"
16
16
-**Cold-start doc improvement** — narrowly improves `AGENTS.md` / `CLAUDE.md`. This skill covers that plus seven more artifacts.
17
-
-**Harness audit** — asks: *is this repo set up so agents can ship without a human babysitting every step?*
17
+
-**This skill** — asks: *is this repo set up so agents can ship without a human babysitting every step?*
When the user asks for readiness but also asks you to change files, prefer the matching `audit+fix` mode.
58
+
59
+
## Baseline: the 8-artifact stack
20
60
21
61
For each artifact, score `✅ Present / ⚠️ Partial / ❌ Missing` with concrete path evidence and a one-line gap.
22
62
@@ -29,14 +69,32 @@ For each artifact, score `✅ Present / ⚠️ Partial / ❌ Missing` with concr
29
69
7.**Repo-scoped skills/prompts** — `.agents/skills/`, `.pi/skills/`, `.claude/skills/`, `.codex/`, `.pi/prompts/`, or package-scoped resources. Prefer 5-10 deep resources over many shallow ones.
30
70
8.**Garbage-collection cadence** — documented recurring loop that converts agent/PR feedback into permanent rules, lints, docs, or skills.
31
71
72
+
## Symphony readiness overlay
73
+
74
+
Use this overlay when the user mentions Symphony, ticket-level orchestration, unattended agents, or preparing repos for an agent scheduler.
75
+
76
+
Score the overlay separately using `references/symphony-readiness.md`:
77
+
78
+
1.**`WORKFLOW.md` contract** — repo-local orchestrator contract with tracker states, workspace hooks, agent settings, and ticket lifecycle prompt.
79
+
2.**Disposable workspace bootstrap** — fresh clone/workspace can install dependencies, prepare services, and clean up without hidden local state.
80
+
3.**One-command validate loop** — stable command for the meaningful quality gate.
81
+
4.**Agent-visible app validation** — app/browser/log/test path an agent can drive and inspect.
82
+
5.**Evidence protocol** — documented proof format for reproduction, validation, screenshots/videos/logs, and PR/ticket handoff.
83
+
6.**Ticket/PR lifecycle skill** — repo-scoped workflow for workpad, branch, PR, review comments, human review, rework, and landing.
84
+
7.**Agent-readable observability** — token-efficient CLI/API access to local logs, CI failures, browser console, metrics, or traces.
If multiple stack markers exist, audit every major stack that has runtime/build impact. For example, a TypeScript frontend plus Python backend should use both stack references and call out cross-stack bootstrap/validation gaps.
53
113
54
-
Read the matching reference from `references/stack-<stack>.md`. If there is no dedicated stack reference, use `references/audit-prompt.md` plus generic judgment.
114
+
Read the matching reference from `references/stack-<stack>.md`. If there is no dedicated stack reference, use `references/audit-prompt.md` plus generic judgment. For `symphony-readiness` or `audit+fix:symphony`, also read `references/symphony-readiness.md`, `references/evidence-protocol.md`, and `references/smoke-ticket-eval.md`.
55
115
56
116
### Phase 3 — Run the audit
57
117
58
118
Use `references/audit-prompt.md` as the checklist. Execute the audit directly with available Pi tools (`read`, `bash`, `rg`/`find` through shell as needed). Keep evidence path-specific.
59
119
60
120
Return:
61
121
62
-
- 8 artifact scorecards with evidence.
122
+
- 8 baseline artifact scorecards with evidence.
63
123
- Summary scorecard (`X/8 ✅, Y/8 ⚠️, Z/8 ❌`).
64
124
- Top 3 highest-leverage gaps with effort estimate and what each unlocks.
65
125
- Notable strengths.
66
126
- Stack-specific observations.
67
127
68
-
Keep the report under about 1500 words. Do not pad.
128
+
For Symphony modes, additionally return:
129
+
130
+
- 9-item Symphony readiness overlay scorecard.
131
+
- Symphony readiness verdict: `Ready`, `Close`, or `Not ready`.
132
+
- Minimum blockers before unattended orchestration.
133
+
- One recommended smoke-ticket eval shape for this repo.
134
+
135
+
Keep the baseline report under about 1500 words. For Symphony modes, keep the combined report under about 2200 words unless the user asks for depth. Do not pad.
69
136
70
137
### Phase 4 — Mode-specific follow-up
71
138
72
139
-**`audit`**: stop after the report. Do not edit.
73
-
-**`audit+fix`**: fix the top 3 gaps, or fewer if the safe scope is smaller.
74
-
-**`focus:<artifact>`**: skip the full audit and handle only that artifact.
140
+
-**`audit+fix`**: fix the top 3 baseline gaps, or fewer if the safe scope is smaller.
141
+
-**`symphony-readiness`**: audit baseline plus Symphony overlay. Do not edit.
142
+
-**`audit+fix:symphony`**: fix the smallest set of blockers that moves the repo toward unattended ticket execution. Prefer workflow contract, disposable bootstrap, validation/evidence, and smoke eval over cosmetic docs.
143
+
-**`focus:<artifact>`**: read-only by default; skip the full audit and audit only that artifact. For focused edits, use the normal `audit+fix` mode and state the artifact scope in natural language, e.g. `audit+fix, focus pre-commit`.
75
144
76
-
Use `references/fix-patterns.md` for fix templates.
145
+
Use `references/fix-patterns.md` for fix templates. For Symphony workflow seeding, use `references/workflow-template.md`. For proof requirements, use `references/evidence-protocol.md` and `references/smoke-ticket-eval.md`.
77
146
78
147
Fix rules:
79
148
80
149
- Make surgical changes only; no unrelated refactors.
81
150
- Verify functionally and show output.
82
151
- Do not commit or push unless the user explicitly requested commits.
83
152
- Do not post to external systems unless explicitly requested.
84
-
- Avoid emojis in committed files unless the repo already uses them.
153
+
- Avoid emojis in committed source/config/docs unless the repo already uses them. Status emojis in audit chat reports are allowed.
85
154
- Use kebab-case filenames unless the repo convention differs.
86
155
- Run fixes sequentially when they touch overlapping files.
Copy file name to clipboardExpand all lines: skills/harness-audit/references/audit-prompt.md
+35-3Lines changed: 35 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,11 @@
1
-
# Audit Prompt Template
1
+
# Audit Checklist / Delegation Prompt
2
2
3
-
Use this as the audit checklist or as a prompt for a separate exploration agent when the host supports one. Replace the `{{...}}` placeholders before using it.
3
+
Use this directly as the audit checklist. Paths in this file are relative to the skill root (`skills/harness-audit/`), not to the `references/` directory. When delegating to a separate exploration agent, first replace every `{{...}}` placeholder with concrete repo values; do not send unresolved placeholders to another agent.
4
+
5
+
Placeholder sources:
6
+
-`{{STACK}}`, `{{PACKAGE_MANAGER}}`, `{{REPO_PATH}}`, and `{{REPO_CONTEXT}}` come from stack detection and initial repo inspection.
7
+
-`{{LINT_CONFIG_PATHS}}`, `{{TEST_RUNNER}}`, and `{{STACK_SPECIFIC_BONUS_SIGNALS}}` come from the matching stack reference.
8
+
-`{{LINT_CMD}}` for fix templates must be derived from package scripts, Makefile targets, or the matching stack reference; if no lint command exists, report that as the gap instead of inventing one.
4
9
5
10
---
6
11
@@ -61,6 +66,22 @@ Concentration check: 5-10 deep skills > 50 shallow ones. Flag sprawl.
61
66
## 8. Garbage collection cadence
62
67
This is hard to detect from code alone. Look for: weekly review docs, `CHANGELOG.md` with structured entries, `docs/` folder with retros/postmortems, `.github/ISSUE_TEMPLATE/` for "agent slop" or similar, `docs/decisions/` ADRs. Note any signal that PR feedback gets converted into rules/lints over time.
63
68
69
+
# Symphony readiness overlay
70
+
71
+
When the requested mode is `symphony-readiness` or `audit+fix:symphony`, also use `references/symphony-readiness.md` and score the repo on:
72
+
73
+
1.`WORKFLOW.md` contract
74
+
2. Disposable workspace bootstrap
75
+
3. One-command validate loop
76
+
4. Agent-visible app validation
77
+
5. Evidence protocol
78
+
6. Ticket/PR lifecycle skill
79
+
7. Agent-readable observability
80
+
8. Safety, secrets, and workspace policy
81
+
9. Smoke-ticket eval
82
+
83
+
This overlay answers whether an unattended ticket-level orchestrator can safely run agents in this repo. Do not treat good docs/tests as sufficient if there is no disposable bootstrap, evidence path, or smoke-ticket proof.
84
+
64
85
# Bonus signals to capture
65
86
66
87
`{{STACK_SPECIFIC_BONUS_SIGNALS}}` — read the matching `references/stack-{{STACK}}.md` for stack-specific things to look for here.
@@ -87,4 +108,15 @@ What's already well-set-up that should be preserved.
87
108
## Stack-specific observations
88
109
Toolchain caveats agents need to know (Swift project file conflicts, Bun gotchas, Python venv conventions, etc.).
89
110
90
-
Keep total report under ~1500 words. Be ruthless — gaps not present, but no padding.
111
+
For Symphony modes, also include:
112
+
113
+
## Symphony readiness overlay
114
+
9-item scorecard with evidence and gaps.
115
+
116
+
## Symphony readiness verdict
117
+
Ready / Close / Not ready.
118
+
119
+
## Minimum blockers before unattended orchestration
120
+
The smallest set of fixes needed before a scheduler can run agents unattended.
121
+
122
+
Keep baseline reports under ~1500 words. Keep Symphony reports under ~2200 words unless depth is requested. Be ruthless — gaps not present, but no padding.
0 commit comments