perf: parallelize gt status to eliminate N+1 bd subprocess bottleneck#3504
Open
mmlac wants to merge 2 commits intogastownhall:mainfrom
Open
perf: parallelize gt status to eliminate N+1 bd subprocess bottleneck#3504mmlac wants to merge 2 commits intogastownhall:mainfrom
mmlac wants to merge 2 commits intogastownhall:mainfrom
Conversation
gt status was taking >60s due to sequential bd subprocess calls — one per agent for hook discovery, plus two per rig for MQ summary. This commit applies four fixes: 1. Batch hook discovery: new FindAllHandoffBeads() fetches all pinned beads in a single bd call, replacing N per-agent subprocess calls. 2. Parallelize within-rig: hook discovery, MQ summary, and agent runtime checks now run concurrently within each rig goroutine. 3. Batch MQ summary: single bd list call with status=all, split open vs in_progress in memory (was 2 separate calls per rig). 4. Parallelize tmux/git in gt rig status: HasSession, CurrentBranch, and git Status checks for polecats and crew run in parallel. Reduces bd subprocess calls from ~45 to ~4 for a typical 3-rig setup. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> Executed-By: mayor
github-actions
bot
added
the
status/needs-triage
The dominant bottleneck was polecatMgr.List() which called loadFromBeads() sequentially for each polecat — each doing 3-6 bd/git subprocess calls. With 7 polecats this took ~56s. Changes: - Parallelize polecat.Manager.List(): load all polecats concurrently via goroutines instead of sequential loop - Restructure runRigStatus to gather ALL data in parallel before display: witness IsRunning, refinery IsRunning+Queue, polecatMgr.List, and crewMgr.List now run concurrently, then HasSession checks run in a second parallel wave Result: gt rig status drops from ~70s to ~20s (limited by slowest single polecat's loadFromBeads). Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> Executed-By: mayor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
FindAllHandoffBeads()fetches all pinned beads in a singlebdcall, replacing the N+1 pattern where each agent triggered a separatebdsubprocess (~27 calls → 1 per rig)sync.WaitGroupbd list --status=allcall split in memory, replacing 2 sequential calls per riggt rig status:HasSession,CurrentBranch, andgit Statuschecks for polecats and crew workers run in parallel goroutinesReduces total
bdsubprocess calls from ~45 to ~4 for a typical 3-rig workspace. Expected improvement from >60s to a few seconds.Files changed
internal/beads/handoff.go— AddedFindAllHandoffBeads()batch methodinternal/cmd/status.go— RewrotediscoverRigHooks()to use batch fetch + in-memory resolution; parallelized intra-rig work; batched MQ queriesinternal/cmd/rig.go— Parallelized polecat and crew status checks inrunRigStatus()Test plan
go build ./...passesgo test ./internal/beads/...passesgt statuscompletes in <10s on a multi-rig workspacegt rig status <rig>completes in <5s with many polecatsgt status --faststill works (skips hooks/MQ as before)gt statusoutput. (note: the hook indicators didn't show for me before this, either)🤖 Generated with Claude Code