feat: bundles phase 1 — Themes: keyword list and theme-keyed cross-repo bundling

[Jammy2211]

Overview

Bundles phase 1. The dashboard's auto-bundler (PyAutoBrain#309) groups on Target: — a mechanical key, so proposals read as "three things that live in autoarray" rather than "three things about MGE". This adds a Themes: keyword list to the prompt header and re-keys auto-bundling on the primary theme (cross-repo), with the remaining keywords as packing affinity. Follow-up to complete/2026/08/dashboard-bundles.md; phase 2 (nightly Claude theme-fill + proposed bundles) is filed separately and blocked on this.

Plan

• Add the optional Themes: list header (same form as Repos:) with a small controlled vocabulary in PyAutoMind/REFERENCE.md; intake assigns it at formalisation.
• Re-key auto_bundles(): first bullet = primary theme = grouping key (each prompt in ≤1 bundle); remaining bullets = Jaccard affinity for packing inside a pool; Target stays the fallback key for un-themed prompts. Card title = primary theme (+ shared secondaries); members show their repo. Unknown keywords warn ⚠️ like unknown epic/bundle slugs.
• One-off Opus backfill sweep writes Themes: (1–3 keywords, primary first) on the ~132 draft/ prompts where clear — one reviewable diff, never overwriting an existing value.
• Tests: primary grouping, affinity packing, cross-repo members, fallback-to-Target, unknown-keyword warning, and the no-themes fixture rendering identically to today.
• Regenerate the dashboard; nightly needs no change.

Detailed implementation plan

Affected Repositories

• PyAutoBrain (primary)
• PyAutoMind

Branch Survey

Repository	Current Branch	Dirty?
./PyAutoBrain	main	clean
./PyAutoMind	main	clean

Suggested branch: feature/bundle-themes

Implementation Steps

1. PyAutoBrain/agents/conductors/intake/_intake.py
   • Parse Themes: as a list (reuse the Repos: list parsing); census records carry themes: list[str].
   • THEME_VOCAB loaded from PyAutoMind/REFERENCE.md (or a small themes.md/YAML — pick whichever the Mind conventions favour; the vocabulary must be human-editable without touching Brain). Unknown keyword → ⚠️ on the card and a hygiene count.
   • auto_bundles(): pool by primary theme when present, else by Target (fallback pools are labelled as today). Within a pool, greedy packing by affinity: seed with the highest-priority member, add the candidate with the highest Jaccard overlap of full theme lists (ties: priority, Target, path) until the existing cap (8 pts, ≤4 members, ≤1 large) is hit; min 2. All existing exclusions unchanged. Slug auto-<theme>-<n>; title <theme> + shared secondaries; members table gains a Repo column.
   • Intake formalise step: prompt for / assign Themes: (keep it optional — never block formalisation on it).
2. PyAutoMind/REFERENCE.md: Themes: in the header docs + the vocabulary table (seed: epic slugs + mge, point-source, jax-compile, jax-gradient, ci-smoke, dashboard, assistants, cti, interferometer, cluster, samplers, docs-hub, …).
3. Backfill sweep (separate subagent, separate commit): read each draft/**/*.md, insert Themes: after Repos: (or after Target:) only where absent and clear; leave unclear prompts untouched; report a table of path → keywords for review.
4. Tests in PyAutoBrain/tests/test_intake_dashboard.py.
5. Regenerate dashboard.md/html; --check clean.

Key Files

• PyAutoBrain/agents/conductors/intake/_intake.py — parse_bundles, auto_bundles, bundle_cards, renderers
• PyAutoBrain/tests/test_intake_dashboard.py
• PyAutoMind/REFERENCE.md, PyAutoMind/draft/**

Original Prompt

Click to expand starting prompt

Bundles phase 1 — Themes: keyword list and theme-keyed (cross-repo) auto-bundling

Type: feature
Target: PyAutoMind
Repos:

• PyAutoMind
• PyAutoBrain
  Difficulty: medium
  Autonomy: supervised
  Priority: high
  Status: formalised
  Filed: 2026-08-27

Original request (verbatim)

what is the premise of grouping?, they feel a bit random but I guess its hard
to know how to group really. it feels like they are grouped on the source code
library rather than the task and its scientific context, where the latter
would generally be more cross repo

i think we want 1 and 2?

could even be multiple key words in bullet points to help more?

Context

The Bundles section shipped 2026-08-27 (record
complete/2026/08/dashboard-bundles.md, PyAutoBrain#309). Its auto-bundler
groups on Target: only — a mechanical key (one worktree per repo), not a
topical one — so proposals read as "three things that live in autoarray", not
"three things about MGE". The useful grouping is scientific/topical and is
routinely cross-repo; the start_bundle contract already allows that (one
shared worktree per repo, parallel across repos).

Scope

1. Themes: header — optional light-header key in the same list form as
   Repos::

   Themes:
   - mge
   - jax-gradient
   - interferometer
   

   Small, controlled, human-editable vocabulary documented in REFERENCE.md
   (seed from the epic slugs and the obvious clusters: mge, point-source,
   jax-compile, jax-gradient, ci-smoke, dashboard, assistants,
   cti, …). Unknown keywords render a ⚠️ like unknown epic/bundle slugs so
   the list never rots into free-text tags. Intake assigns them at
   formalisation going forward (_intake.py formalise step).

2. Renderer (PyAutoBrain/agents/conductors/intake/_intake.py
auto_bundles) — deterministic, keyed on themes, cross-repo allowed:

   • first bullet = primary theme = the grouping key; every prompt lands in
     at most one auto bundle (no duplicates across cards);
   • remaining bullets = affinity: within a primary-theme pool, packing
     prefers members with the highest keyword overlap (Jaccard over the whole
     list), so a large pool splits by what the work is about rather than by
     filename order; ties broken by priority, then Target, then path;
   • Target remains the fallback grouping key for prompts with no themes;
   • keep every existing exclusion, the size cap, min 2, the top-8 display and
     pinned/auto ordering. Card title = primary theme (+ the shared secondary
     keywords, if any); members show their repo.

3. One-off backfill — an Opus sweep reads each of the ~132 draft/ prompts
   and writes a Themes: list where clear (1–3 keywords, primary first; leave
   absent when unclear); reviewable as a single diff, committed by the human.
   Never overwrite an existing value.

4. Tests for primary-theme grouping, affinity-driven packing, cross-repo
   members, fallback-to-Target, unknown-keyword warning, and that the existing
   Target-only fixture still renders identically when no prompt carries themes.

Deferred to phase 2 (bundle_nightly_claude_pass.md): assigning theme lists to
new drafts automatically and Claude-proposed cross-theme bundles — the
affinity keywords are exactly the signal that pass uses.

[Jammy2211]

Library PR Created

• PyAutoBrain: feat(intake): Themes: keyword list and theme-keyed cross-repo bundling #312 — Themes: parsing, theme-pooled affinity packing, Repo column, intake classify --themes; 96/96 intake, 618/618 full suite.
• PyAutoMind: mind: themes.md vocabulary, Themes: backfill, themed dashboard PyAutoMind#366 — themes.md (24 keywords), REFERENCE docs, spawn/ledger rules, Themes: backfilled on 130/136 drafts, regenerated dashboard; 281/281.

Result: 23 auto bundles, 22 theme-keyed; every rendered card spans 2–3 repos. Merge order: Brain first (Mind's dashboard check renders with Brain main). API changes: none (optional Themes: header). Heart RED for unrelated reasons; shipped on explicit human acknowledgement. Next: /prm.

[Jammy2211]

Shipped

Bundles phase 1 is merged — auto-bundles are now theme-keyed and cross-repo, not Target:-keyed.

• PyAutoBrain#312 — Themes: parsing, theme-primary pooling with Jaccard affinity packing, unknown-keyword ⚠️, intake classify --themes, 12 new tests (618 Brain tests pass).
• PyAutoMind#366 — themes.md controlled vocabulary (24 keywords), REFERENCE.md, spawn/ledger rules, workflow triggers, Themes: backfilled on 130/136 drafts, regenerated dashboard.

Behaviour: the first keyword is the primary theme and the grouping key (every prompt lands in at most one bundle); the remaining keywords are packing affinity, so a large pool splits by what the work is about rather than by filename order. Target: stays the fallback for un-themed prompts, and an un-themed Mind renders byte-identically to before (pinned by test_an_unthemed_backlog_renders_exactly_as_it_did_before_themes).

Live result: 22 theme-keyed bundles + 1 Target fallback; the rendered top 8 all span 2–3 repos — pixelization (autoarray + workspaces), point-source (autogalaxy + autolens + pyautolens), ci-smoke, notebooks, samplers · jax-gradient (autofit + autolens_profiling), visualization, jax-compile.

Phase 2 (nightly Claude pass: theme-fill new drafts, propose cross-theme bundles) stays filed as draft/feature/pyautomind/bundle_nightly_claude_pass.md — the affinity keywords are exactly the signal it consumes.

Vocabulary gap noted for later: no keyword yet for PyAutoMemory / knowledge-wiki work.