Tracking: CI flaky-test stability — timing/state-machine races across agent, codemap, and MCP shards

[moonray]

Context

A sample of 18 recent failed CI runs (9 on main, 9 on PRs) shows a systemic pattern: timing / state-machine races that pass on a warm local machine but fail intermittently under hosted-runner load (macOS 26.4). This issue tracks them as a group so the CI red signal can be trusted again and so otherwise-green PRs stop getting blocked by noise.

Dominant flake

• CI flake: AgentRunWorktreeStartTests/testProductionCapturePreservesBoundStoredSelectionInsteadOfCanonicalUI fails intermittently on shard 4 #420 — AgentRunWorktreeStartTests/testProductionCapturePreservesBoundStoredSelectionInsteadOfCanonicalUI — 11/18 failures, 4 of them on main. Race: "the launching selection changed while its frozen review capability was being created." On its own this is the majority of recent red CI.

Codemap async / state-machine cluster

• CodeMapArtifactBuildCoordinatorTests.testHookOverflowDropsNewestPreservesAcceptedFIFOAndDrainsWithoutLeak (2×)
• CodemapAutomaticSelectionBusyTests.testAutomaticSelectionRetriesBusySourceTwiceThenBecomesReady
• CodemapGraphFreezeQueryTests.testPresentationRenderFailsClosedAfterDemandCancellationCatalogAdvanceAndUnload
• CodemapAutomaticSelectionColdTests.testAutomaticSelectionFinalizationDeadlineFailsClosedWhileCleanupContinues
• CodemapAutomaticSelectionBasicTests.testSecondFlushRecoveryRetiresFlightAndCoalescesSignalsIntoOneSuccessor

Partly in flight: #349 (hermetic git fixtures / hang-proof codemap engine tests).

MCP / worktree timing cluster

• MCPCodeStructureWorktreeTests.testSelectedScopeRejectsAtLimitPlusOneWithoutContentOrCodemapWork
• MCPAskOracleWorktreeTests.testAgentRunVisibleLinkedWorktreeFreshAndContinuingOracleInheritsLaunchArtifact
• MCPAskOracleWorktreeTests.testAskOraclePackagesMultipleBoundRootsWithoutCanonicalLeak

Common root

Every one of these is a concurrency / timing / state-machine test — FIFO drain, busy retry, deadline/cancellation, flush recovery, worktree freshness, frozen-capability-vs-selection races. They reproduce locally but flake on hosted runners under load. Same class of defect as #420.

Notes

• morluto has an in-flight morl/stabilizeci branch (no PR yet) — likely relevant to this effort.
• Sample is 18 of ~38 recent failed runs. The dominant flake is unmistakable; the Codemap cluster is a clear secondary signal; the MCP ones are rarer (1× each).

Suggested approach

• Stabilize the timing-sensitive seams (deterministic setup + explicit await/quiesce before asserting) instead of relying on wall-clock timing.
• Until fixed, consider quarantining or auto-retrying the worst offender (CI flake: AgentRunWorktreeStartTests/testProductionCapturePreservesBoundStoredSelectionInsteadOfCanonicalUI fails intermittently on shard 4 #420) so it stops blocking otherwise-green PRs (e.g. Title fresh Codex threads on the direct streaming path #370 is red solely because of it).

(Filing without labels — filer lacks label permission on this repo. Maintainer please add e.g. bug + area:agent-mode / area:codemap / area:mcp.)