test: decide and wire a testing strategy for grimoire's executable code (#34) - #53
test: decide and wire a testing strategy for grimoire's executable code (#34)#53Jartans-Familiar wants to merge 2 commits into
Conversation
code-hygiene bundled comment truthfulness with dead code, reinvention, naming and faking-done, so a diff that only needed the comment rules did not reliably auto-activate the skill (#39). Comments, tombstone and retold-fact detection, and comment density now live in their own comment-hygiene skill; find-duplicate-comments.js moves with them. Cross-references updated across code-structure, readable-code, docs-patterns and general-reviewer (frontmatter skills: list plus the Important/Minor bullets that were comment-specific). Content is moved, not rewritten -- artifact-only, no new doctrine. praxis bumped 1.4.0 -> 1.5.0 (new skill, minor per RELEASING.md). Codex face: neither .claude-plugin/plugin.json nor .codex-plugin enumerates skills, so no manifest entry to add; general-reviewer.toml regenerated because its source .md body and skills: list changed. Refs JAR-413 Co-authored-by: multica-agent <github@multica.ai>
…de (#34) grimoire's Node hooks/reviewer tooling and scripts/generate-codex.py ran zero assertions; defects were caught only by hand during review (#33's review rounds: an off-by-one in diff line bookkeeping, a path-quoting false-clean, a CRLF false-clean, a submodule abort). Decision, per #34's open questions: - Node: node:test + node:assert/strict. Ships with the Node already running every hook -- no new dependency, no package.json. Tests live beside their source as *.test.js, node:test's own zero-config discovery. - Python: stdlib unittest, not pytest -- same reasoning .pre-commit-config.yaml already gives for skipping ruff (a pyproject.toml to hold config that nothing else in the repo needs). Tests live in scripts/ as test_*.py (unittest's discovery default) and load generate-codex.py via importlib, since its hyphenated filename blocks a plain import. - Two conventions, not one: each stays idiomatic to its own runtime's zero-config discovery rather than fighting either tool to force a shared pattern. - Scope: functions carrying real parsing or bookkeeping logic -- find-duplicate-comments.js's exported pure functions (already seamed for this in #33 and orphaned since) and generate-codex.py's frontmatter parser, qualify/display_name/short_description, and the TOML string escaper. Thin hook wrappers that only call back into the Claude Code hook API are out of scope -- a fixture would assert little. The filesystem-walking functions (collect_targets, find_orphans, find_divergent_copies) run end to end against the real repo on every `make verify --check`, which is itself their regression coverage. - Wiring: `make test`, a new `test` CI job (needs actions/setup-node alongside setup-python), and both the needs list and the results array of ci.yml's check aggregator per its own hand-sync note. Demonstrated locally (not committed, restored before this commit): reintroducing the historical no-newline-marker off-by-one in findRetoldInDiff fails `keeps line numbers correct across a "no newline at end of file" marker`; swapping basic_string's escape order fails `test_escaping_a_backslash_and_a_quote_together_stays_parseable`. Both red before the revert, both green after. praxis bumped 1.5.0 -> 1.5.1 (PATCH: the new test file is non-behavioural, co-located under a skill directory that already ships as plugin content). Codex face: general-reviewer.toml etc. regenerated, no content changed under them so the diff is the version line only. Refs JAR-413 Co-authored-by: multica-agent <github@multica.ai>
|
Holding this one. #34 was decided and closed by #50, merged as
Also: The gate itself collides outright -- both branches define #52 and #54 implement the merged convention: 139 Node tests and 40 Python tests, the four hook process contracts, and the four What is worth keeping from here is the coverage, not the wiring. If any of the 40 cases in this branch assert something #52 misses, port those cases onto |
|
resolve conflicts |
Closes #34.
Stacked on #51 (
agent/loom/1ea9712f) -- the Node test file lives besidefind-duplicate-comments.jsat its post-split location(
plugins/praxis/skills/comment-hygiene/), so this PR's base is thatbranch rather than
main. Merge #51 first; this one will retargetmainautomatically once it does.
Decision (the questions #34 asked)
node:test+node:assert/strict. Ships with the Node alreadyrunning every hook -- zero new dependency, no
package.json/lockfile toadd. Tests colocate with source as
*.test.js,node:test's ownzero-config discovery pattern (
node --testwith no path args finds themrecursively from repo root).
unittest, not pytest -- the same reasoning.pre-commit-config.yamlalready states for skipping ruff: apyproject.tomlto hold config that nothing else in the repo needs. Tests live in
scripts/as
test_*.py(unittest discover's default pattern) and loadgenerate-codex.pyviaimportlib.util, since its hyphenated filenamecan't be a plain
import.own runtime's zero-config discovery rather than fighting either tool to
force a shared filename convention across languages.
find-duplicate-comments.js's exported pure functions (prose,buildSkipMatcher,findRetoldInDiff,dedupePairs-- seamed for this infeat: repo hygiene enforcement, plus praxis compact and skill-activation fixes #33, orphaned since, per Decide a testing strategy for the marketplace's executable code #34's own description) get 19 cases covering the
three defect classes feat: repo hygiene enforcement, plus praxis compact and skill-activation fixes #33's review rounds caught by hand: the diff
line-number off-by-one, the unescaped-
--skip-pattern false-clean, and theCRLF false-clean.
generate-codex.py'sparse_frontmatter,qualify,display_name,short_description,basic_string(the TOML escaper) andrender_agent_tomlget 21 cases.Out of scope, named rather than silently dropped: the thin hook
wrappers (
check-console-log.js,suggest-compact.js,suggest-skills.js,sync-codex-agents.js) -- each is mostly a call into the Claude Code hookAPI, where a fixture would assert little; and
generate-codex.py'sfilesystem-walking functions (
collect_targets,find_orphans,find_divergent_copies,main), which already run end to end against thereal repo on every
make verify --check-- that IS their regressioncoverage, and duplicating it under a mock filesystem would test the mock,
not the logic.
make test, a newtestjob inci.yml(needsactions/setup-nodealongside the existingsetup-python), and both theneeds:list and theresultsarray of thecheckaggregator, per thehand-sync note that file already carries.
CLAUDE.md,CONTRIBUTING.mdand
README.mdall updated somake testis part of the statedpre-PR/pre-done checklist, not a step only CI knows about.
Demonstrated: red before, green after
Both done locally against this branch, then reverted before committing --
neither seeded defect is in the diff:
Node -- removed the "no newline at end of file" marker guard in
findRetoldInDiff(the exact historical off-by-one #34 names):Restored ->
node --test: 19/19 pass.Python -- swapped
basic_string's escape order (quote before backslashinstead of after), which lets a TOML parser read the output as ending the
string early rather than as an escaped quote:
Restored ->
python3 -m unittest discover: 21/21 pass.Domain-owner clause
Artifact-only. This is test infrastructure and CI wiring for code Loom
already owns test coverage for per the role spec; no skill or agent
content changes.
Manifest / Codex face
praxisbumped1.5.0->1.5.1(PATCH: the new test file isnon-behavioural -- it ships as plugin content because it's colocated under
skills/comment-hygiene/, but nothing at runtime reads it). Codex face:general-reviewer.tomland the plugin manifests regenerated byscripts/generate-codex.py; nothing underplugins/praxis/agents/orcodex/changed content-wise in this PR, so the regen diff is the versionline only.
Verification
Task JAR-413.