Impact
CI app shard 2 can fail on an Agent Mode tool-tracking concurrency test, blocking otherwise unrelated PR validation noise and making it hard to distinguish product regressions from test instability.
Observed behavior
On PR #388, exact head a1b5fbe8, Build and Test (app shard 2) failed twice on the same test after a one-time rerun:
AgentToolTrackingControllerTests/testConcurrentRawUnregisterAndStopJoinCapturedDeliveryBarrier
- Failure location:
Tests/RepoPromptTests/AgentMode/AgentToolTrackingControllerTests.swift:227
- Assertion: expected
firedCount == 1, observed firedCount == 0
- Distinctive log excerpt:
[ClaudeToolTracking] MCP observer call skipped no event observers
The failed workflow was the PR CI run for head a1b5fbe8; all other CI jobs for that head completed successfully.
Expected behavior
The test should deterministically capture the intended delivery-barrier event, or the contract should be rewritten so the ordering it asserts is guaranteed by the implementation.
Evidence suggesting this is unrelated to PR #388
PR #388 changes MCP agent-run/worktree inheritance and startup-pending wait snapshot behavior. It does not touch:
Tests/RepoPromptTests/AgentMode/AgentToolTrackingControllerTests.swift
AgentToolTrackingController
- the tool-tracking observer registration/unregistration implementation path implicated by the log excerpt
A local focused run of the failing test passed:
make dev-test FILTER='AgentToolTrackingControllerTests/testConcurrentRawUnregisterAndStopJoinCapturedDeliveryBarrier'
That combination points to a timing/order-sensitive CI flake or latent concurrency issue rather than a regression in the PR's changed code path.
Reproduction notes
- Run PR CI app shard 2 on a head that includes the current test suite.
- Observe whether
AgentToolTrackingControllerTests/testConcurrentRawUnregisterAndStopJoinCapturedDeliveryBarrier fails with firedCount == 0.
- Compare against a focused local run of the same test.
Frequency from the observed PR: 2/2 hosted shard-2 attempts failed after the initial failure and one rerun; focused local run passed once.
Acceptance criteria
- The test no longer fails because observer delivery sees an empty observer map under the captured-delivery-barrier scenario, or the test is adjusted to assert only behavior guaranteed by the production contract.
- A focused run of
AgentToolTrackingControllerTests/testConcurrentRawUnregisterAndStopJoinCapturedDeliveryBarrier passes.
- CI shard 2 can pass without requiring repeated reruns for this specific test.
Impact
CI app shard 2 can fail on an Agent Mode tool-tracking concurrency test, blocking otherwise unrelated PR validation noise and making it hard to distinguish product regressions from test instability.
Observed behavior
On PR #388, exact head
a1b5fbe8,Build and Test (app shard 2)failed twice on the same test after a one-time rerun:AgentToolTrackingControllerTests/testConcurrentRawUnregisterAndStopJoinCapturedDeliveryBarrierTests/RepoPromptTests/AgentMode/AgentToolTrackingControllerTests.swift:227firedCount == 1, observedfiredCount == 0The failed workflow was the PR CI run for head
a1b5fbe8; all other CI jobs for that head completed successfully.Expected behavior
The test should deterministically capture the intended delivery-barrier event, or the contract should be rewritten so the ordering it asserts is guaranteed by the implementation.
Evidence suggesting this is unrelated to PR #388
PR #388 changes MCP agent-run/worktree inheritance and startup-pending wait snapshot behavior. It does not touch:
Tests/RepoPromptTests/AgentMode/AgentToolTrackingControllerTests.swiftAgentToolTrackingControllerA local focused run of the failing test passed:
make dev-test FILTER='AgentToolTrackingControllerTests/testConcurrentRawUnregisterAndStopJoinCapturedDeliveryBarrier'That combination points to a timing/order-sensitive CI flake or latent concurrency issue rather than a regression in the PR's changed code path.
Reproduction notes
AgentToolTrackingControllerTests/testConcurrentRawUnregisterAndStopJoinCapturedDeliveryBarrierfails withfiredCount == 0.Frequency from the observed PR: 2/2 hosted shard-2 attempts failed after the initial failure and one rerun; focused local run passed once.
Acceptance criteria
AgentToolTrackingControllerTests/testConcurrentRawUnregisterAndStopJoinCapturedDeliveryBarrierpasses.