t18649: Docker cleanup: owner labels, post-merge teardown and pulse reaping for disposable test stacks #34285

Worker Guidance

Files to Modify

• NEW: .agents/scripts/docker-resource-helper.sh — subcommands labels, compose-labels, inventory, teardown --worktree <path>, reap [--apply] [--max-seconds N], report. Model the dry-run/--force split and logging on .agents/scripts/system-cleanup.sh.
• EDIT: .agents/scripts/pulse-cleanup.sh:1174-1179 — in cleanup_stale_opencode, after the system-cleanup.sh call, run docker-resource-helper.sh reap --max-seconds "${PULSE_DOCKER_CLEANUP_MAX_SECONDS:-30}" (plus --apply only when AIDEVOPS_DOCKER_REAP=1), output to $LOGFILE, non-fatal (|| true). Same -x guard as the system-cleanup.sh block. Do not hook into pulse-cleanup-worktree-removal.sh: it is 2,024 lines, already over the 1,500-line shell file-size gate.
• EDIT: .agents/scripts/full-loop-helper-merge-cleanup.sh:494-500 — after _merge_remove_worktree_for_cleanup succeeds, run docker-resource-helper.sh teardown --worktree "$worktree_path" non-fatally (|| true), only when docker is on PATH and the daemon answers.
• EDIT: .agents/scripts/wp-plugin-release-helper.sh:997 — add the owner labels to the plugin-check network, volume and containers it creates (keep wprh_pc_cleanup).
• NEW: .agents/scripts/tests/test-docker-resource-helper.sh — PATH-stubbed docker (no real daemon) covering the acceptance criteria.
• NEW: .agents/tools/containers/disposable-resources.md — how to start a labelled disposable site (docker run/Compose examples using the helper), teardown, the report, and the address-pool fix.
• EDIT: .agents/tools/containers/orbstack.md — one Troubleshooting line pointing to the new doc for the address-pool error and cleanup.
• EDIT: .agents/reference/domain-index.md:67 — add tools/containers/disposable-resources.md to the Infrastructure row.

Files Scope

• .agents/scripts/docker-resource-helper.sh
• .agents/scripts/pulse-cleanup.sh
• .agents/scripts/full-loop-helper-merge-cleanup.sh
• .agents/scripts/wp-plugin-release-helper.sh
• .agents/scripts/tests/test-docker-resource-helper.sh
• .agents/tools/containers/disposable-resources.md
• .agents/tools/containers/orbstack.md
• .agents/reference/domain-index.md
• .agents/scripts/worktree-helper-integration.sh
• .agents/scripts/worktree-helper-cmds.sh
• .agents/scripts/worktree-clean-lib.sh

Integration scope recovery (worker, GH#34285): _merge_cleanup_linked_worktree in full-loop-helper-merge-cleanup.sh has no callers on the default branch (rg -n _merge_cleanup_linked_worktree .agents/scripts returns only its definition); cmd_merge always records a deferred cleanup receipt, and the linked worktree is removed later through worktree-helper.sh remove (_remove_cleanup_and_execute -> _remove_finalize_post_removal in worktree-helper-cmds.sh, which already frees localdev/preview resources from worktree-helper-integration.sh). The post-merge Docker teardown therefore lives in that shared post-removal hook (a guarded docker_resource_auto_teardown beside preview_proxy_auto_free), covering merge cleanup and manual removal; the pulse's post-merge removal runs worktree-helper.sh clean --auto --force-merged (pulse-cleanup-worktree-state.sh), whose removal path in worktree-clean-lib.sh (beside localdev_auto_branch_rm) gets the same one-line guarded call. Code file size is not gated (linters-local.sh: only Markdown size is ratcheted). full-loop-helper-merge-cleanup.sh is left unchanged because its helper already routes through worktree-helper.sh remove. Collision check: prework-discovery-helper.sh found no open related PRs.

Complete Write Surface

• Callers/readers: pulse preflight cleanup_stale_opencode in .agents/scripts/pulse-cleanup.sh:1161; post-merge _merge_cleanup_linked_worktree in .agents/scripts/full-loop-helper-merge-cleanup.sh:475; plugin-check stack in .agents/scripts/wp-plugin-release-helper.sh:926-1070; agents reading tools/containers/*.md via .agents/reference/domain-index.md:67. Search evidence: rg -n -i docker over system-cleanup.sh, pulse-cleanup.sh, pulse-cleanup-fixture-orphans.sh, full-loop-helper-merge-cleanup.sh returns nothing today.
• Writers/mutation paths: docker rm -f, docker network rm, docker volume rm, only through docker-resource-helper.sh and only on resources carrying sh.aidevops.owner=1. Existing writer kept as-is: wprh_pc_cleanup (wp-plugin-release-helper.sh:932-936).
• Existing verification/tests: .agents/scripts/tests/test-full-loop-merge-worktree-cleanup.sh (merge cleanup), .agents/scripts/tests/test-pulse-wrapper-characterization.sh (pulse stage list, includes cleanup_stale_opencode). New .agents/scripts/tests/test-docker-resource-helper.sh with a PATH-stubbed docker is justified because the reap rule deletes data and a stub is the cheapest way to prove the negative cases without a daemon.
• Schemas/config: label keys are the schema; env AIDEVOPS_DOCKER_REAP (default 0), AIDEVOPS_DOCKER_TTL_HOURS (default 72), PULSE_DOCKER_CLEANUP_MAX_SECONDS (default 30). No configs/*.json change.
• Generated/deployed mirrors: setup.sh --non-interactive deploys .agents/scripts/ and .agents/tools/ to ~/.aidevops/agents/; no generated files.
• Migrations/backfills: N/A — no stored state; existing unlabelled resources are never labelled retroactively and are only reported. Evidence: the labels are new, so nothing reads them before this change.
• Cleanup/rollback paths: revert the PR; labels already on resources are inert without the helper. Leaving AIDEVOPS_DOCKER_REAP unset (default) disables removal by the pulse without a revert; post-merge teardown can be disabled with AIDEVOPS_DOCKER_TEARDOWN=0.

Implementation Steps

1. Write docker-resource-helper.sh with labels/compose-labels first, then inventory (labelled resources as TSV: kind, id, name, repo, ref, worktree, created, ttl, state), report, teardown, reap.
2. Ownership rule for reap: remove a labelled resource only if (a) its sh.aidevops.worktree path no longer exists and its sh.aidevops.ref issue/PR is closed/merged, or (b) now - created > ttl-hours and it has no running container. Remove containers before networks and volumes; skip a network or volume still attached to any container not being removed.
3. Bound GitHub lookups: one gh call per distinct ref per cycle, cached for the cycle; on API failure, skip (never treat unknown as closed).
4. Add the pulse call in cleanup_stale_opencode; report-only unless AIDEVOPS_DOCKER_REAP=1.
5. Add teardown to post-merge cleanup.
6. Label the plugin-check stack in wp-plugin-release-helper.sh.
7. Write the doc, the orbstack line and the index entry.
8. WIP commit, then run verification.

Hazards and Compatibility

• Concurrency/atomicity: another session may be starting or using a stack. Never reap a resource created in the last 30 minutes; re-check running state and the worktree path immediately before each removal; a pulse run and a post-merge teardown racing on the same object both tolerate "No such container/network/volume".
• Migration/rollback: no data migration. Ordering inside a removal: containers first, then networks, then volumes; skip a network or volume still attached to any container that is not being removed. Rollback is a PR revert; removed volumes are not recoverable, which is why reaping is report-only by default.
• Mixed-version/backward compatibility: older checkouts and ad-hoc commands don't add labels; their resources stay unmanaged (reported only), exactly as today. Docker without label support is not a concern (labels exist since Docker 1.6).
• Idempotency/retry: removing an already-removed object is ignored; repeated runs converge. A failed gh ref lookup means "keep" for this cycle, retried next cycle.
• Partial failure/recovery: if removal stops part-way (timeout, daemon restart), the next run re-inventories from labels and finishes; no local state file to corrupt. Docker missing or the daemon down: every entry point returns 0 with one log line, using a short docker info timeout, so pulse and merge never fail or slow because of Docker.
• Data loss: volumes may hold a test site someone still wants. Mitigations: owner label required, worktree-gone and ref-closed (or TTL with nothing running), report-only default, inventory shows exactly what would go.
• Portability and budget: follow reference/bash-compat.md (Bash 3.2: no mapfile, no GNU-only date); honour --max-seconds like system-cleanup.sh.

Verification Before Dispatch

.agents/scripts/linters-local.sh --changed
bash .agents/scripts/tests/test-docker-resource-helper.sh
bash .agents/scripts/tests/test-full-loop-merge-worktree-cleanup.sh
bash .agents/scripts/tests/test-pulse-wrapper-characterization.sh
.agents/scripts/docker-resource-helper.sh report

• Surface mapping: linters-local.sh --changed → ShellCheck and markdown lint for every file in Files Scope (style, local var="$1", explicit returns). test-docker-resource-helper.sh → docker-resource-helper.sh ownership rule, report-only default, unlabelled/open-ref/lookup-failure negatives, daemon-down exit 0 (acceptance criteria 1, 3, 4, 5, 6; Concurrency and Partial-failure hazards). test-full-loop-merge-worktree-cleanup.sh → full-loop-helper-merge-cleanup.sh still removes worktrees with Docker absent (criteria 2, 6). test-pulse-wrapper-characterization.sh → pulse-cleanup.sh still exposes the cleanup_stale_opencode stage the pulse schedules (criterion 6). docker-resource-helper.sh report on a machine with Docker → real inventory, no deletions (criterion 3).
• Manual check (machine with Docker): start a labelled container + network + volume with a temporary worktree path, delete the path, close-state stub or TTL 0, run reap (lists it) then reap --apply (removes it); an unlabelled container beside it is untouched.
• Broad verification trigger: Not required — no shared config, root tooling or release infrastructure changes.

Recoverability Checkpoint

WIP commit after step 7, before running the broader checks.

Safety-Stop Recovery

If removal logic cannot be made safe within the PR, ship labels, inventory, report, teardown and docs with reaping left report-only, keep this issue open, and record the remaining reaping criteria in a comment.

Task Brief

What

Give aidevops ownership and cleanup for the Docker resources that sessions and scripts create (disposable WordPress/app test sites, plugin-check stacks, review/verification sites), so they are removed once the work that needed them is finished:

1. Owner labels. A new docker-resource-helper.sh prints the --label arguments (and a Compose labels: snippet) that mark a container, network or volume as aidevops-owned: sh.aidevops.owner=1, sh.aidevops.repo=<owner/repo>, sh.aidevops.ref=<GH#N or PR#N>, sh.aidevops.worktree=<path>, sh.aidevops.created=<epoch>, sh.aidevops.ttl-hours=<N> (default 72). Docs and framework scripts that start disposable stacks use it.
2. Teardown after merge. When the post-merge cleanup removes a linked worktree, it also removes the labelled containers, networks and volumes whose sh.aidevops.worktree matches it.
3. Regular cleanup. A pulse step removes labelled resources whose worktree is gone and whose linked issue/PR is closed/merged, or whose TTL has expired. It runs report-only until AIDEVOPS_DOCKER_REAP=1 (default 0 for the first release). Resources without sh.aidevops.owner=1 are never removed by the pulse; they are only counted in the report.
4. Pressure report. docker-resource-helper.sh report prints counts and sizes of unused volumes, images and build cache, networks with no containers, and containers stopped longer than the TTL, with the safe commands to reclaim each. It deletes nothing.
5. Address-pool guidance. Document the all predefined address pools have been fully subnetted error: find networks with no containers, remove only those, and optionally set Docker's default-address-pools (for example {"base":"10.200.0.0/16","size":24}) for about 256 networks instead of about 30.

Why

Observed on one maintainer Mac (2026-10-10), with no aidevops Docker cleanup in place:

• docker network create failed with all predefined address pools have been fully subnetted. This blocked a plugin release's update test and every repo smoke/plugin-check script that starts a disposable site, in all sessions.
• 94 containers, about 40 of them leftover test sites running or stopped for 3–10 days. The running ones held about 2.5 GB RAM. Six were named after issues already closed.
• docker system df: 1,139 volumes with 184 GB reclaimable, 78 GB of reclaimable images, 34 GB of build cache.
• None of the containers carried an owner label, so a session could not tell which were finished. Cleanup needed a person to match names to issues by hand.

aidevops already cleans up worktrees, branches, stashes, workspace scratch and orphaned test-server processes (GH#33747), but nothing touches Docker: no Docker calls exist in system-cleanup.sh, pulse-cleanup.sh, pulse-cleanup-fixture-orphans.sh or full-loop-helper-merge-cleanup.sh. The one framework script that starts containers, wp-plugin-release-helper.sh, removes its own on exit (wprh_pc_cleanup, line 932), which is the pattern to keep. Ad-hoc sites that agents start for verification have no equivalent.

Tier

Tier checklist (verify before assigning)

• Exact execution contract supplied? No: helper interface and pulse step need design.
• Targets and reference pattern verified?
• No semantic or design decision remains? No: label schema details, issue/PR-closed lookup budget, TTL default and reaping safety rules need decisions.
• Bounded, reversible, low-consequence impact? No: removes containers and volumes (data loss if wrong).
• No stateful coordination to invent? No: merge cleanup and pulse must agree on ownership.
• Focused verification and rollback are explicit?
• No dispatch-path risk override? No: pulse-cleanup.sh is listed in .agents/configs/self-hosting-files.conf.

Selected tier: tier:thinking

Tier rationale: deletes user data (volumes) based on an ownership rule that must be designed, and edits the pulse dispatch path (pulse-cleanup.sh is a self-hosting file).

Context & Decisions

• Labels, not name prefixes: names are free-form and ambiguous; labels are what Docker filters on (--filter label=...).
• Report-only first: the first release proves the ownership rule on real machines before anything is deleted automatically.
• Unlabelled resources are never auto-removed. Interactive sessions may remove them only with the user's say-so.
• Out of scope: image and build-cache pruning (report only), remote Docker hosts, non-Docker runtimes.

Brief Workflow

This issue body is composed under .agents/workflows/brief.md. Newly queued auto-dispatch work must pass its Dispatch Readiness Contract (brief schema v2): complete write surface, hazards and compatibility, executable verification mapped to affected surfaces, and positive plus negative/regression acceptance criteria.