Parent
#172
Summary (trimmed)
This ticket keeps only the model-only fusion hardening work and removes the already-validated pi-subagents discussion from scope.
Scope (current)
1) Context engineering for panel/judge/synthesis
Fusion currently reuses broad, repeated context across all pipeline stages:
- panel stage gets full rendered conversation per model,
- judge stage reuses all raw panel text,
- synthesis stage again receives all raw panel text + judge output.
This creates redundant token spend and noisy inputs.
2) Judging the merged outcome
The pipeline currently judges panel responses, then synthesizes, but there is no automated judge of the final merged answer in the bench/report path.
Proposed work
- Add bounded/selected context forwarding for panel members (cheap context-selector defaults, e.g. full vs
last_user_turn/diff-focused slices).
- Prefer structured panel output shape where possible (
{claims,evidence,risks,confidence}), with robust free-text fallback parsing.
- Forward to synthesis only what is needed: judge findings + selected panel artifacts, rather than full raw panel text.
- Add an automated rubric-style judge step for the merged synthesis answer in
scripts/bench-fusion.ts, alongside existing manual scores.
- Emit a low-trust flag when synthesis judge confidence is weak/automated rubric fails.
Out-of-scope (trimmed away)
pi-subagents for kind=fusion execution (already handled as model-level parallelism).- Future
kind=agent_fusion implementation.
- Command-surface changes.
Why this is now
This keeps the ticket focused on immediate, high-signal, low-noise fusion hardening in the current model-only runner.
Acceptance
- Reduced unnecessary token forwarding per run is measured and observable in bench output.
- Bench includes an automated merged-output quality signal (in addition to manual scores).
- No behavior changes for
kind=agent_fusion (not yet implemented).
- No regressions in existing fusion success/error handling.
Tests
- Extend
packages/fusion/src/runner.test.ts for:
- structured panel output parsing fallback,
- selective synthesis context forwarding.
- Add benchmark regression coverage for new
--report rubric metric in scripts/bench-fusion.ts (mock-backed).
Non-goals
- No subagent command/surface changes.
- No changes outside fusion scope in this ticket.
Parent
#172
Summary (trimmed)
This ticket keeps only the model-only fusion hardening work and removes the already-validated
pi-subagentsdiscussion from scope.Scope (current)
1) Context engineering for panel/judge/synthesis
Fusion currently reuses broad, repeated context across all pipeline stages:
This creates redundant token spend and noisy inputs.
2) Judging the merged outcome
The pipeline currently judges panel responses, then synthesizes, but there is no automated judge of the final merged answer in the bench/report path.
Proposed work
last_user_turn/diff-focused slices).{claims,evidence,risks,confidence}), with robust free-text fallback parsing.scripts/bench-fusion.ts, alongside existing manual scores.Out-of-scope (trimmed away)
pi-subagentsforkind=fusionexecution (already handled as model-level parallelism).kind=agent_fusionimplementation.Why this is now
This keeps the ticket focused on immediate, high-signal, low-noise fusion hardening in the current model-only runner.
Acceptance
kind=agent_fusion(not yet implemented).Tests
packages/fusion/src/runner.test.tsfor:--reportrubric metric inscripts/bench-fusion.ts(mock-backed).Non-goals