Skip to content

Add completely (Claude Code plugin harness with a default-FAIL evaluator in the loop's close path) - #40

Open
23ag1 wants to merge 1 commit into
walkinglabs:mainfrom
23ag1:add-completely
Open

Add completely (Claude Code plugin harness with a default-FAIL evaluator in the loop's close path)#40
23ag1 wants to merge 1 commit into
walkinglabs:mainfrom
23ag1:add-completely

Conversation

@23ag1

@23ag1 23ag1 commented Jun 11, 2026

Copy link
Copy Markdown

What this adds

One entry to Runtimes, Harnesses & Reference Implementations (placed next to Citadel, the other Claude Code harness):

completely β€” a Claude Code plugin harness focused on the verification half of harness engineering: done is earned, not asserted.

Why it fits this list's bar

  • Specific about how agents are constrained, evaluated, resumed, and orchestrated (the contributing criteria):
    • Constrained β€” deterministic PreToolUse/PostToolUse hooks: per-edit lint/type gate, dangerous-command block, edit-time write-zone fencing per task, commit-before-close gate, and reviewer findings that block closing while unresolved.
    • Evaluated β€” an independent, read-only, default-FAIL evaluator sits in the close path of the autonomous loop; every acceptance criterion stays FAIL until evidence is re-run (with path-exercised / exercised-as-a-user / code-read dimensions and an adversarial claim-vs-refute mode).
    • Resumed β€” claims carry worker-id + heartbeat; an interrupted run's orphaned tasks are auto-reopened at next start (the loop's run-report never reports a half-done run as finished).
    • Orchestrated β€” parallel fresh-context claude -p workers over disjoint write-zones (a hardened take on the Ralph loop already on this list), with a post-batch integration gate that checks the union composes.
  • Original implementation, inspectable β€” the harness design lives in plain bash/python + markdown contracts; behaviour is pinned by a 77-test contract suite the repo runs on itself.
  • Link verified, no duplicate entry, single-line diff per the entry format.

Happy to adjust placement or tighten the description if you prefer.

@23ag1

23ag1 commented Jun 11, 2026

Copy link
Copy Markdown
Author

Added a primary-source write-up that may help evaluate the entry: Done is earned β€” a field report on putting a default-FAIL evaluator in the close path. It documents four real incidents from the harness's own build log (a crash that shipped with a fully green self-test, the evaluator rejecting its own verification fix twice, a watchdog false-killing healthy workers, and a safety gate that passed its self-test while dead in production) and the evaluator dimension each one forced. Happy to fold it into the PR as a second entry under Evals & Observability if you think it fits β€” otherwise the single Runtimes entry stands as-is.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant