-
Notifications
You must be signed in to change notification settings - Fork 9
Add open source infrastructure and dogfood CLAUDE.md #9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,77 @@ | ||
| --- | ||
| generated: 2026-03-24 | ||
| level: 3 | ||
| level_name: Enforced | ||
| score: 17 | ||
| total: 37 | ||
| stack: node-javascript | ||
| monorepo: false | ||
| pillars: | ||
| style-validation: { pass: 0, total: 4 } | ||
| testing: { pass: 3, total: 5 } | ||
| git-hooks: { pass: 2, total: 5 } | ||
| documentation: { pass: 6, total: 9 } | ||
| agent-config: { pass: 0, total: 5 } | ||
| code-quality: { pass: 3, total: 3 } | ||
| dev-environment: { pass: 1, total: 3 } | ||
| agentic-workflow: { pass: 2, total: 2 } | ||
| --- | ||
|
|
||
| # Harness Readiness Report | ||
|
|
||
| **Project:** harness-engineering (Node.js / Claude Code plugin) | ||
| **Level:** 3 / 5 (Enforced) | ||
| **Score:** 17 / 36 criteria passing | ||
|
|
||
| ## Pillar Scores | ||
|
|
||
| Style & Validation ░░░░░░ 0/4 | ||
| Testing ████░░ 3/5 | ||
| Git Hooks ██░░░░ 2/5 | ||
| Documentation ████░░ 6/9 | ||
| Agent Configuration ░░░░░░ 0/5 | ||
| Code Quality ██████ 3/3 | ||
| Dev Environment ██░░░░ 1/3 | ||
| Agentic Workflow ██████ 2/2 | ||
|
|
||
| ## Passing | ||
|
|
||
| - ✓ Test runner configured (Jest, `node --experimental-vm-modules jest tests/scripts/`) | ||
| - ✓ Test files mirror source files (`tests/scripts/` mirrors `skills/setup/scripts/`) | ||
| - ✓ Tests pass (5 unit test files, eval suite with 3 fixtures) | ||
| - ✓ Secret scanning configured (check-secrets.js, runs in CI workflow) | ||
| - ✓ File size limits enforced (check-file-sizes.js, runs in CI workflow) | ||
| - ✓ CLAUDE.md exists (108 lines, well-scoped) | ||
| - ✓ Has Commands section (6 commands, copy-paste ready) | ||
| - ✓ Has Architecture section (tree, key modules table, data flow) | ||
| - ✓ Has Critical Gotchas section (5 project-specific, non-obvious discoveries) | ||
| - ✓ No drift (all 11+ documented paths verified to exist on disk) | ||
| - ✓ Content quality (dense, specific, actionable, no generic boilerplate) | ||
| - ✓ No source files over 300 lines (all scripts under limit) | ||
| - ✓ No hardcoded secrets in source (only intentional one in eval fixture) | ||
| - ✓ Consistent code style across codebase | ||
| - ✓ Build/dev commands documented and functional | ||
| - ✓ Workflow system present (/readiness and /setup skills with structured phases) | ||
| - ✓ Session-start validation documented (validate-docs.js referenced in CLAUDE.md) | ||
|
|
||
| ## Failing | ||
|
|
||
| - ✗ Linter not configured for this repo (ships ESLint templates to users but no eslint config at root) | ||
| - ✗ Formatter not configured for this repo (ships Prettier template but no .prettierrc at root) | ||
| - ✗ Lint-on-commit not configured (no lint-staged for this repo) | ||
| - ✗ No default exports rule not enforced (no ESLint running) | ||
| - ✗ Coverage threshold not configured | ||
| - ✗ TDD enforcement rule missing (no .claude/rules/tdd.md at root) | ||
| - ✗ Pre-commit hook not installed (.git/hooks/ has only .sample files) | ||
| - ✗ Pre-push hook not installed (template exists but not wired) | ||
| - ✗ Smart test caching not active (template has SHA-based caching but not installed) | ||
| - ✗ Quality gates not documented in CLAUDE.md (limits embedded in table, not summarized) | ||
| - ✗ Code review checklist missing from CLAUDE.md | ||
| - ✗ AUTO markers not used (repo doesn't dogfood its own auto-generation feature) | ||
| - ✗ .claude/settings.json does not exist at repo root | ||
| - ✗ Allow list not configured (no settings.json) | ||
| - ✗ Deny list not configured (no settings.json) | ||
| - ✗ Path-scoped rules missing (no .claude/rules/ at root) | ||
| - ✗ Enforcement hierarchy incomplete (CI + prose, but no path-scoped rules layer) | ||
| - ✗ .env.example missing at root | ||
| - ✗ No package.json at root (dependencies not formally declared) | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| root = true | ||
|
|
||
| [*] | ||
| indent_style = space | ||
| indent_size = 2 | ||
| end_of_line = lf | ||
| charset = utf-8 | ||
| trim_trailing_whitespace = true | ||
| insert_final_newline = true | ||
|
|
||
| [*.md] | ||
| trim_trailing_whitespace = false | ||
|
|
||
| [Makefile] | ||
| indent_style = tab |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| # Default owner for everything | ||
| * @jrenaldi79 | ||
|
|
||
| # Skills require explicit review | ||
| skills/ @jrenaldi79 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| --- | ||
| name: Bug Report | ||
| about: Report a problem with the plugin, skills, or enforcement scripts | ||
| title: '' | ||
| labels: bug | ||
| assignees: '' | ||
| --- | ||
|
|
||
| ## Description | ||
|
|
||
| A clear description of the bug. | ||
|
|
||
| ## Steps to Reproduce | ||
|
|
||
| 1. | ||
| 2. | ||
| 3. | ||
|
|
||
| ## Expected Behavior | ||
|
|
||
| What you expected to happen. | ||
|
|
||
| ## Actual Behavior | ||
|
|
||
| What actually happened. Include error messages or screenshots if applicable. | ||
|
|
||
| ## Environment | ||
|
|
||
| - **OS**: (e.g., macOS 15, Ubuntu 24.04, Windows 11) | ||
| - **Node.js version**: (e.g., v22.x) | ||
| - **Claude Code version**: (e.g., v2.1.x) | ||
| - **Plugin version**: (from `.claude-plugin/plugin.json`) | ||
|
|
||
| ## Additional Context | ||
|
|
||
| Any other context about the problem. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| --- | ||
| name: Feature Request | ||
| about: Suggest a new feature or improvement | ||
| title: '' | ||
| labels: enhancement | ||
| assignees: '' | ||
| --- | ||
|
|
||
| ## Problem | ||
|
|
||
| What problem does this solve? What's the current limitation? | ||
|
|
||
| ## Proposed Solution | ||
|
|
||
| Describe what you'd like to happen. | ||
|
|
||
| ## Alternatives Considered | ||
|
|
||
| Any alternative approaches you've thought about. | ||
|
|
||
| ## Additional Context | ||
|
|
||
| Any other context, examples, or references. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| ## Summary | ||
|
|
||
| Brief description of the changes. | ||
|
|
||
| ## Changes | ||
|
|
||
| - | ||
|
|
||
| ## Testing | ||
|
|
||
| - [ ] Unit tests pass: `node --experimental-vm-modules node_modules/.bin/jest tests/scripts/` | ||
| - [ ] Evals pass (if skill logic changed): `bash tests/evals/run-evals.sh` | ||
| - [ ] README updated (if adding features or changing behavior) | ||
| - [ ] CLAUDE.md updated (if adding/removing/renaming files in `skills/` or `tests/`) | ||
|
|
||
| ## Related Issues | ||
|
|
||
| Closes # |
| Original file line number | Diff line number | Diff line change | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,69 @@ | ||||||||||
| name: CI | ||||||||||
|
|
||||||||||
| on: | ||||||||||
| push: | ||||||||||
| branches: [main] | ||||||||||
| pull_request: | ||||||||||
| branches: [main] | ||||||||||
|
|
||||||||||
| jobs: | ||||||||||
| test: | ||||||||||
| runs-on: ubuntu-latest | ||||||||||
| strategy: | ||||||||||
| matrix: | ||||||||||
| node-version: [18, 20, 22] | ||||||||||
|
|
||||||||||
| steps: | ||||||||||
| - uses: actions/checkout@v4 | ||||||||||
|
|
||||||||||
| - name: Use Node.js ${{ matrix.node-version }} | ||||||||||
| uses: actions/setup-node@v4 | ||||||||||
| with: | ||||||||||
| node-version: ${{ matrix.node-version }} | ||||||||||
|
|
||||||||||
| - name: Install dependencies | ||||||||||
| run: npm ci --ignore-scripts | ||||||||||
|
|
||||||||||
| - name: Run unit tests | ||||||||||
| run: node --experimental-vm-modules node_modules/.bin/jest tests/scripts/ --ci | ||||||||||
|
|
||||||||||
| - name: Validate plugin manifest | ||||||||||
| run: | | ||||||||||
| node -e " | ||||||||||
| const fs = require('fs'); | ||||||||||
| const plugin = JSON.parse(fs.readFileSync('.claude-plugin/plugin.json', 'utf8')); | ||||||||||
| if (!plugin.name || !plugin.version || !plugin.description) { | ||||||||||
| console.error('Invalid plugin.json: missing required fields'); | ||||||||||
| process.exit(1); | ||||||||||
| } | ||||||||||
| console.log('plugin.json is valid:', plugin.name, 'v' + plugin.version); | ||||||||||
| " | ||||||||||
|
|
||||||||||
| - name: Check for secrets in repo | ||||||||||
| run: node skills/setup/scripts/lib/check-secrets.js --all || true | ||||||||||
|
Comment on lines
+42
to
+43
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Secret scanning is non-enforcing and effectively ineffective in CI. Line [43] masks failures with Minimum workflow-side fix- - name: Check for secrets in repo
- run: node skills/setup/scripts/lib/check-secrets.js --all || true
+ - name: Check for secrets in repo
+ run: node skills/setup/scripts/lib/check-secrets.js --allYou should also ensure 📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||||||
|
|
||||||||||
| - name: Verify no files exceed 300 lines | ||||||||||
| run: | | ||||||||||
| node -e " | ||||||||||
| const fs = require('fs'); | ||||||||||
| const path = require('path'); | ||||||||||
| const dirs = ['skills', 'tests/scripts']; | ||||||||||
| let violations = 0; | ||||||||||
| function walk(dir) { | ||||||||||
| for (const entry of fs.readdirSync(dir, { withFileTypes: true })) { | ||||||||||
| const full = path.join(dir, entry.name); | ||||||||||
| if (entry.isDirectory() && entry.name !== 'node_modules' && entry.name !== 'fixtures') { | ||||||||||
| walk(full); | ||||||||||
| } else if (entry.isFile() && /\.(js|ts|md)$/.test(entry.name)) { | ||||||||||
| const lines = fs.readFileSync(full, 'utf8').split('\n').length; | ||||||||||
| if (lines > 300 && !full.includes('SKILL.md') && !full.includes('README.md') && !full.includes('global-claude.md')) { | ||||||||||
|
Comment on lines
+58
to
+59
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Line-count gate has an off-by-one risk at the 300-line boundary. Line [58] counts Suggested fix- const lines = fs.readFileSync(full, 'utf8').split('\n').length;
- if (lines > 300 && !full.includes('SKILL.md') && !full.includes('README.md') && !full.includes('global-claude.md')) {
- console.error(full + ': ' + lines + ' lines (max 300)');
+ const content = fs.readFileSync(full, 'utf8');
+ const rawLines = content.split('\n').length;
+ const lines = content.endsWith('\n') ? rawLines - 1 : rawLines;
+ if (lines > 300 && !full.includes('SKILL.md') && !full.includes('README.md') && !full.includes('global-claude.md')) {
+ console.error(full + ': ' + lines + ' lines (max 300)');
violations++;
}🤖 Prompt for AI Agents |
||||||||||
| console.error(full + ': ' + lines + ' lines (max 300)'); | ||||||||||
| violations++; | ||||||||||
| } | ||||||||||
| } | ||||||||||
| } | ||||||||||
| } | ||||||||||
| dirs.forEach(d => { if (fs.existsSync(d)) walk(d); }); | ||||||||||
| if (violations > 0) process.exit(1); | ||||||||||
| console.log('All files within line limits'); | ||||||||||
| " | ||||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| # Changelog | ||
|
|
||
| All notable changes to this project will be documented in this file. | ||
|
|
||
| The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). | ||
|
|
||
| ## [1.0.0] - 2026-03-24 | ||
|
|
||
| ### Added | ||
|
|
||
| - `/readiness` skill: evaluates codebases across 8 pillars and 5 maturity levels with scored reports | ||
| - `/setup` skill: scaffolds CLAUDE.md files, enforcement scripts, git hooks, linter configs, and agent settings | ||
| - Enforcement scripts: secret scanning, file size limits, test colocation, doc generation, drift detection | ||
| - Git hook templates: pre-commit (6 checks) and pre-push (test suite with SHA caching) | ||
| - CLAUDE.md templates: global (cross-project standards) and project (per-codebase guidance) | ||
| - Path-scoped rule templates: TDD, code quality, testing, TypeScript | ||
| - Config templates: ESLint, Prettier, lint-staged, .gitignore, .env.example, Claude Code settings | ||
| - Reference mapping of 20+ best practices from OpenAI, Anthropic, Augment Code, Factory.ai, and practitioners | ||
| - Evaluation suite with 3 fixture projects (level 1, 3, 5) and automated grading | ||
| - Unit tests for all setup scripts | ||
| - Plugin manifest for Claude Code marketplace |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,108 @@ | ||
| # CLAUDE.md | ||
|
|
||
| This file provides guidance to Claude Code when working with code in this repository. | ||
|
|
||
| ## Project Overview | ||
|
|
||
| **harness-engineering** is a Claude Code plugin and reference for AI coding agent harnesses. It provides two skills (`/readiness` and `/setup`) that analyze and configure projects for agent-assisted development, plus a README mapping 20+ best practices from industry sources. | ||
|
|
||
| ### Core Features | ||
|
|
||
| - **`/readiness`**: Scores a codebase across 8 pillars and 5 maturity levels, produces a saved report with delta tracking | ||
| - **`/setup`**: Scaffolds CLAUDE.md files, enforcement scripts, git hooks, linter configs, and agent settings via Socratic questioning | ||
| - **Reference guide**: Maps best practices from OpenAI, Anthropic, Augment Code, Factory.ai, and practitioners to concrete implementation patterns | ||
|
|
||
| --- | ||
|
|
||
| ## Essential Commands | ||
|
|
||
| ### Testing | ||
| ```bash | ||
| node --experimental-vm-modules node_modules/.bin/jest tests/scripts/ # Unit tests for setup scripts | ||
| bash tests/evals/run-evals.sh # E2E evals against fixture projects | ||
| node tests/evals/grader.js # Grade eval results | ||
| bash tests/evals/test-marketplace-install.sh # Test plugin install flow | ||
| ``` | ||
|
|
||
| ### Validation | ||
| ```bash | ||
| node skills/setup/scripts/lib/generate-docs.js --check # Verify auto-generated sections are current | ||
| node skills/setup/scripts/lib/validate-docs.js --full # Check for documentation drift | ||
| ``` | ||
|
|
||
| --- | ||
|
|
||
| ## Architecture | ||
|
|
||
| ``` | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Add language labels to fenced code blocks (MD040). Lines [37] and [59] start unlabeled fenced blocks. markdownlint flags these; add Suggested doc fix-```
+```text
harness-engineering/
...
-```
+```
-```
+```text
User installs plugin
...
-```
+```Also applies to: 59-59 🧰 Tools🪛 markdownlint-cli2 (0.21.0)[warning] 37-37: Fenced code blocks should have a language specified (MD040, fenced-code-language) 🤖 Prompt for AI Agents |
||
| harness-engineering/ | ||
| ├── .claude-plugin/ # Plugin manifest (plugin.json, marketplace.json) | ||
| ├── skills/ | ||
| │ ├── readiness/ | ||
| │ │ └── SKILL.md # Readiness analysis skill (8 pillars, 5 levels) | ||
| │ └── setup/ | ||
| │ ├── SKILL.md # Setup orchestrator skill (6 phases) | ||
| │ ├── scripts/ # Node.js enforcement scripts installed into target projects | ||
| │ │ ├── lib/ # Individual checks (secrets, file sizes, test colocation, docs) | ||
| │ │ └── hooks/ # Git hook scripts (pre-commit, pre-push) | ||
| │ ├── templates/ # Config templates (.prettierrc, eslint, CLAUDE.md, rules/) | ||
| │ └── references/ # Stack patterns, enforcement docs, quality guide | ||
| ├── tests/ | ||
| │ ├── scripts/ # Unit tests for setup scripts (Jest) | ||
| │ └── evals/ # E2E evaluation suite with 3 fixture projects (level 1/3/5) | ||
| ├── assets/ # Pipeline diagram SVG, social images | ||
| └── README.md # Reference guide and documentation | ||
| ``` | ||
|
|
||
| ### Data Flow | ||
|
|
||
| ``` | ||
| User installs plugin | ||
| -> /readiness reads templates/references as benchmark | ||
| -> 3 parallel subagents evaluate project against 8 pillars | ||
| -> Scored report saved to .claude/readiness-report.md | ||
|
|
||
| User runs /setup | ||
| -> Socratic questions determine stack and goals | ||
| -> Scripts scaffold project structure, configs, hooks | ||
| -> Enforcement scripts copied to target project's scripts/ | ||
| -> Git hooks wired to run enforcement on every commit/push | ||
| ``` | ||
|
|
||
| --- | ||
|
|
||
| ## Key Modules | ||
|
|
||
| | Module | Purpose | | ||
| |--------|---------| | ||
| | `skills/readiness/SKILL.md` | Full evaluation framework, 8 pillars, 37 criteria, 3 subagent dispatch | | ||
| | `skills/setup/SKILL.md` | 6-phase setup orchestrator with Socratic discovery | | ||
| | `scripts/init-project.js` | Node/TS project scaffolding (package.json, tsconfig, directories) | | ||
| | `scripts/install-enforcement.js` | Copies enforcement scripts, hooks, configs into target project | | ||
| | `scripts/generate-claude-md.js` | Generates tailored CLAUDE.md from templates | | ||
| | `scripts/lib/check-secrets.js` | Pattern-matches API keys, tokens, private keys in staged files | | ||
| | `scripts/lib/check-file-sizes.js` | Rejects files over 300 lines | | ||
| | `scripts/lib/check-test-colocation.js` | Verifies source files have colocated test files | | ||
| | `scripts/lib/generate-docs.js` | Auto-regenerates CLAUDE.md sections between AUTO markers | | ||
| | `scripts/lib/validate-docs.js` | Detects drift between source code and CLAUDE.md content | | ||
|
Comment on lines
+80
to
+87
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Key Modules paths are inconsistent with the documented repo structure. Lines [80]-[87] reference Suggested doc fix-| `scripts/init-project.js` | Node/TS project scaffolding (package.json, tsconfig, directories) |
-| `scripts/install-enforcement.js` | Copies enforcement scripts, hooks, configs into target project |
-| `scripts/generate-claude-md.js` | Generates tailored CLAUDE.md from templates |
-| `scripts/lib/check-secrets.js` | Pattern-matches API keys, tokens, private keys in staged files |
-| `scripts/lib/check-file-sizes.js` | Rejects files over 300 lines |
-| `scripts/lib/check-test-colocation.js` | Verifies source files have colocated test files |
-| `scripts/lib/generate-docs.js` | Auto-regenerates CLAUDE.md sections between AUTO markers |
-| `scripts/lib/validate-docs.js` | Detects drift between source code and CLAUDE.md content |
+| `skills/setup/scripts/init-project.js` | Node/TS project scaffolding (package.json, tsconfig, directories) |
+| `skills/setup/scripts/install-enforcement.js` | Copies enforcement scripts, hooks, configs into target project |
+| `skills/setup/scripts/generate-claude-md.js` | Generates tailored CLAUDE.md from templates |
+| `skills/setup/scripts/lib/check-secrets.js` | Pattern-matches API keys, tokens, private keys in staged files |
+| `skills/setup/scripts/lib/check-file-sizes.js` | Rejects files over 300 lines |
+| `skills/setup/scripts/lib/check-test-colocation.js` | Verifies source files have colocated test files |
+| `skills/setup/scripts/lib/generate-docs.js` | Auto-regenerates CLAUDE.md sections between AUTO markers |
+| `skills/setup/scripts/lib/validate-docs.js` | Detects drift between source code and CLAUDE.md content |🤖 Prompt for AI Agents |
||
|
|
||
| --- | ||
|
|
||
| ## Critical Gotchas | ||
|
|
||
| - **SKILL.md is the skill**: Claude Code reads the SKILL.md file as the skill prompt. Changes to SKILL.md directly change skill behavior. | ||
| - **Scripts run in target projects, not this repo**: The enforcement scripts in `skills/setup/scripts/` are templates copied into user projects by `/setup`. They must work standalone with zero dependencies on this repo. | ||
| - **Eval fixtures are intentionally broken**: `tests/evals/fixtures/level-1-bare/` contains a hardcoded secret on purpose for detection testing. Do not "fix" it. | ||
| - **No package.json at root**: This is a Claude Code plugin, not an npm package. Tests run via direct node/jest/bash invocation. | ||
| - **`globs:` not `paths:`**: Rule files use `globs:` in YAML frontmatter for path scoping. The official docs say `paths:` but `globs:` works more reliably (see Claude Code issue #17204). | ||
|
|
||
| --- | ||
|
|
||
| ## Docs Map | ||
|
|
||
| | Topic | File | | ||
| |-------|------| | ||
| | CLAUDE.md quality criteria | `skills/setup/references/claude-md-guide.md` | | ||
| | Enforcement script patterns | `skills/setup/references/enforcement-scripts.md` | | ||
| | Node/TypeScript stack reference | `skills/setup/references/stack-node-typescript.md` | | ||
| | Eval suite documentation | `tests/evals/README.md` | | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fix the inconsistent total criteria count.
Line [6] says
total: 37, but Line [24] says17 / 36. Keep one source of truth to avoid confusion in reporting.Suggested doc fix
Also applies to: 24-24
🤖 Prompt for AI Agents