feat(backends): add a beads backend behind the Store seam - #52
Open
LeonidShamis wants to merge 5 commits into
Open
feat(backends): add a beads backend behind the Store seam#52LeonidShamis wants to merge 5 commits into
LeonidShamis wants to merge 5 commits into
Conversation
Adds BeadsStore, a Store implementation backed by the beads issue tracker (https://github.com/gastownhall/beads) through the bd CLI: - bd --json subprocesses with a per-invocation list cache; one `bd list --json --all` serves every read - state maps queued/in_flight/done onto open/in_progress/closed; bd-native blocked/deferred statuses read back as queued because blocked/held stay derived above the seam - kind, repo, priority, holds, canonical dates, dependency reasons, and meta live in a tasks_axi object in bd issue metadata; a hold with --until also sets bd's defer date - links stay folded into the title prose exactly like the markdown backend, so deriveLinks works identically on both - after every mutation the backend rewrites the configured markdown path as a read-only canonical mirror, byte-compatible with the markdown grammar, so direct backlog.md readers keep working - prune archives surplus Done tasks to done-archive.md and hides them via a metadata flag while beads retains the full record - public-followups are unsupported and now capability-gated in the command layer with a structured UNSUPPORTED error Shared Task normalization moves from the markdown backend into src/backends/normalize.ts so both backends validate identically. Config grows a [beads] table (dir, bin); context resolution becomes a backend map. Real-bd conformance tests skip when bd is absent.
Installs a pinned @beads/bd on the Linux CI leg so the BeadsStore suite exercises the real CLI instead of self-skipping, and hardens the test helper's bd init with --non-interactive --skip-hooks --skip-agents and an explicit prefix so a CI environment cannot stall or write git hooks.
All three failing checks on PR kunchenguid#52 (CI, Guard generated files, Require no-mistakes) concluded action_required with zero jobs executed: this is GitHub's fork-PR approval gate for a first-time outside contributor, not a code failure. Every CI step passes locally on Linux with bd@1.2.2 (build, lint, REQUIRE_BD=1 test: 456 passed / 1 skipped, skill check), and the guard script's condition holds (no generated files touched). Record the trap in AGENTS.md so future sessions verify locally and hand the run approval to a maintainer instead of hunting for a code fix.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What Changed
src/backends/beads.ts, a secondStoreimplementation that drives thebdCLI via--jsonsubprocesses with a per-invocation list cache, stores fields beads lacks intasks_aximetadata, and rewrites a canonical-markdown mirror after every mutation;--backend beadsand a[beads]config table (dir,bin) select and configure it throughresolveConfig/resolveTasksContext.src/backends/markdown.tsinto the sharedsrc/backends/normalize.ts, gatedpublic-followupcommands on a new backend capability, and made the generic single-idmvfallback reject moving a task that still blocks active dependents before it creates-then-removes.bdconformance suite (test/backends/beads.test.ts+test/beads-helpers.ts) that self-skips whenbdis absent, and a Linux CI step that installs a pinned@beads/bdand setsREQUIRE_BD=1so the suite cannot silently skip there; README and AGENTS.md document the new backend.Risk Assessment
✅ Low: All seven accepted round-1 fixes are verified implemented and correct — the hold/--defer fix and the bd metadata replace-vs-merge semantics were confirmed empirically against real bd 1.2.2, the locking/rollback/CI-guard changes close their races as claimed, and only two info-level robustness notes remain.
Testing
Ran the 27-test beads conformance suite against a real bd 1.2.2 under REQUIRE_BD=1 (all pass), captured an end-to-end CLI transcript demonstrating the beads backend as a user experiences it — including raw bd evidence that hold --until no longer demotes an in-flight task — behaviorally verified the mv-fallback pre-check, word-bounded LOCKED classifier, fail-closed mutation locking, and REQUIRE_BD fail-loud guard, confirmed the create/prune rollback orderings landed as decided, and ran 185 targeted regression tests on the refactored markdown/config/command surfaces; everything passed and the worktree was left clean.
Evidence: CLI transcript: beads backend end-to-end (real bd 1.2.2)
Evidence: Canonical markdown mirror written by the beads backend
Evidence: Review-fix behavioral checks: mv fallback, LOCKED classifier, lock fail-closed
bd stderr "Error: tst-1 is blocked by tst-2" -> AxiError code=UNKNOWN bd stderr "Error: dependency of type blocks already exists" -> AxiError code=UNKNOWN bd stderr "Error: database is locked" -> AxiError code=LOCKED bd stderr "Error: cannot write: another process holds the beads db" -> AxiError code=LOCKED $ tasks-axi mv blocker-b1 --to /tmp/beads-hard-target.md error: "Task "blocker-b1" is still blocking active tasks: dependent-d2" (VALIDATION_ERROR, exit 2; target file never created) $ tasks-axi add lk-one (with backlog.md.lock held) -> error: backlog is locked by another tasks-axi process (LOCKED); succeeds after releasePipeline
Updates from git push no-mistakes
⏭️ **intent** - skipped
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
src/backends/beads.ts:745- Holding an in-flight task with --until silently demotes it to queued. update() pushes--defer <date>without re-asserting-s, and bd 1.2.2 flips an in_progress issue's status todeferredwhen --defer is set (verified in a scratch workspace); stateFromStatus mapsdeferredback to queued, so aftertasks-axi hold <id> --reason r --until <date>on an in-flight task, the store and mirror report the task as queued while the markdown backend keeps it in flight — wrong state with exit 0, and the hold confirmation still prints the stale in_flight record. Fix (verified against bd 1.2.2): push-s BD_STATUS[task.state]in the samebd updatewhenever--deferis pushed;-s in_progress --defer <date>together keeps in_progress. Add an in-flight hold case to the conformance suite.src/backends/beads.ts:611-bd create --id X --forcesilently overwrites an existing issue (verified with bd 1.2.2: exit 0, title and foreign metadata replaced). The only duplicate guard is an unlocked check-then-act against a cachedbd listsnapshot, so two concurrent tasks-axi creates (or a tasks-axi create racing a bd-native create) with the same id destroy the loser without any error — unlike the markdown backend, whose mutations run under withLock and fail closed. --force is required for the id-prefix bypass, so consider serializing beads mutations under withLock(mirrorPath) to close the tasks-axi-vs-tasks-axi race, and documenting the residual bd-native race.src/commands/state.ts:730- The non-markdown single-id mv fallback (newly reachable via the beads backend) creates the task in the target markdown file and only then removes it from the source; BeadsStore.remove rejects a task that still blocks active dependents, somv A --to other.mdwhere A blocks an active B leaves A present in BOTH backlogs, and a retry after unblocking hits CONFLICT ('already exists in the destination backlog'), requiring manual cleanup. Pre-check active dependents (mirror requireNoActiveDependents) at the command layer before creating in the target so the failure happens before any write.src/backends/beads.ts:895- prune appends the full archive block for all surplus tasks before the per-taskbd update --metadata archivedloop and has no restore on failure. If a mid-loop bd invocation fails (e.g. db busy), the archive already contains entries for tasks that are still active, and a retry re-appends those tasks' lines, producing duplicate archive entries — the markdown backend's prune captures an archive restore point and rolls the append back on persist failure. Either flag first and append only for successfully-flagged tasks, or append per-task after its flag succeeds.src/backends/beads.ts:624- create() is three or more bd invocations (create, optional status update, one dep-add per dep) with no rollback: if the-supdate or a dep add fails afterbd createsucceeded, the task exists half-created in bd, create() throws, and a retry reports CONFLICT 'already exists', stranding the user. bd create supports--deps 'type:id'(verified in bd 1.2.2 help), which would fold the dep step into the create call and shrink the window; reasons could still ride in the initial --metadata payload.src/backends/beads.ts:320- The LOCKED classifier /lock|locked|busy|another process/i substring-matches 'blocked' and 'blocks' — routine words in a dependency tracker's error output (the bd edge type used by this backend is literally 'blocks'). A bd error like a dependency-cycle message mentioning 'blocks' gets mislabeled 'the beads database is busy' with a 'retry once the concurrent bd operation finishes' hint, steering agents to retry a deterministic failure. Use word-bounded / db-specific patterns, e.g. /database is locked|\block\b|\bbusy\b|another process/i (\block\b does not match 'blocked'/'blocks').src/backends/beads.ts:385- taskOrder runs issues.find(...) twice per comparison inside the sort comparator, making loadAll O(n² log n) over the full issue list on every read. Precompute a Map<string, created_at> from issues before sorting.test/beads-helpers.ts:11- The Linux 'real bd coverage' the CI commit adds is unenforced: BD_AVAILABLE silently skips the entire conformance suite whenbd versionfails, so a broken @beads/bd install or a PATH regression on the runner turns the suite into a silent skip while CI stays green. An opt-in guard (e.g. a REQUIRE_BD env var set in the Linux CI job that fails the suite when bd is absent) would make the guarantee fail loud.🔧 Fix: harden beads backend holds, locking, rollback, and CI guard
2 infos still open:
src/backends/beads.ts:899- addDep/removeDep write the dep edge and its reason metadata in two bd calls with no retry convergence for the second: ifbd update --metadatafails afterbd dep addsucceeded (src/backends/beads.ts:899-904), the reason is lost and a retry returns the idempotent 'already' result (false) without backfilling it; symmetrically, removeDep (beads.ts:919-927) can leave a staledep_reasonsentry that silently resurrects if the same edge is later re-added without a reason. Writing the metadata before the edge mutation would make both operations retry-convergent (a reason for a nonexistent edge is ignored on read, so the intermediate state is invisible). Impact is limited to the optional reason annotation and requires a mid-operation bd failure.src/backends/lock.ts:23- The shared lock helper's 30s staleness threshold (src/backends/lock.ts:23) was tuned for millisecond markdown mutations; a beads mutation legitimately holding the mirror lock through several Dolt-backed bd invocations (60s timeout each, create chains 3+) can exceed 30s, so a contending process's LOCKED error shows the 'lock looks stale ... remove <path>.lock' hint while the holder is still alive. The hint's guard ('If no tasks-axi process is running') keeps it safe when followed; if desired, BeadsStore.mutate could pass a larger staleMs via withLock's existing options parameter.✅ **Test** - passed
✅ No issues found.
REQUIRE_BD=1 pnpm vitest run test/backends/beads.test.ts— full beads conformance suite against real bd 1.2.2 (27/27 pass, including the hold --until in-flight regression)End-to-end CLI demo in a scratchbd initworkspace:tasks-axi add/list/hold --until/ready --include-held/done --pr --jsonwith--backend beads, plus mirror backlog.md andbd list --allpersisted statebd show ship-login --jsonafterhold --untilon an in-flight task — status staysin_progress, notdeferredtasks-axi mv blocker-b1 --to <target>on the beads backend with an active dependent — refused with VALIDATION_ERROR and no target file createdLOCKED classifier behavioral check via the publicbinseam with a stub bd: "blocked"/"blocks" stderr → UNKNOWN, "database is locked"/"another process" → LOCKEDMutation lock fail-closed: mutation with a freshbacklog.md.lockheld → LOCKED error, succeeds after releaseREQUIRE_BD fail-loud:PATHshadowed with a broken bd +REQUIRE_BD=1→ suite fails with the actionable install error (exit 1) instead of skipping; inspected.github/workflows/ci.ymlLinux job wiringCode inspection: create() does status-before-deps with documented partial-create recovery; prune() flags each task in bd before appending its archive blockpnpm vitest run test/backends/markdown.test.ts test/commands/state.test.ts test/commands/public-followup.test.ts test/config.test.ts— 185/185 pass on the surfaces this change refactoredMarkdown-backend parity check for duplicateadd(already: truecontract matches beads)✅ **Document** - passed
✅ No issues found.
package.json- prettier is a devDependency but has no config file, no format script, and no CI check, and 7 files untouched by this change (src/backends/lock.ts, src/toon.ts, test/commands/home.test.ts, test/commands/setup.test.ts, test/config.test.ts, test/derive.test.ts, test/release-ci-exclusions.test.ts) fail prettier defaults. This change's own six new/modified files were formatted in this pass; the pre-existing drift was left alone as out of scope. A follow-up could either add a prettier config + check (and format the stragglers) or drop the dependency, so the intended formatter is unambiguous.✅ **Push** - passed
✅ No issues found.