Skip to content

sequential plans lose transitive dependency context #221

Closed

Description

@Shearerbeard

Problem

Tasks in a sequential plan only receive the previous step's result. flatten_sequential (crates/aura/src/orchestration/types.rs) sets each task's dependencies to the step immediately before it, and build_task_context (crates/aura/src/orchestration/orchestrator.rs) only walks direct dependencies. Failure propagation breaks the same way: blocked_tasks only blocks direct dependents when a task fails, so transitive dependents stay pending and the executor's deadlock breaker silently drops them. The test suite documents the limitation as-is ("Only direct dependents are blocked").

Real plan from the SRE hard benchmark (answerbook/ai-experiments aura-e2e, branch mshearer/aura-e2e-tests), restart-investigation prompt on gpt-5.2:

plan:
  T0  query prometheus for pods with restart_count > 5
  T1  fetch alertmanager rules
  T2  for each pod found (T0), determine which rule (T1) would fire

dependencies the plan needs:    T2 <- {T0, T1}
dependencies the runtime built: T2 <- {T1}      (frontier replaced per step)

execution:
  T0  ok    15s   4 tool calls
  T1  ok    11s   2 tool calls
  T2  FAIL  300s  0 tool calls   prompt has T1's rules but not T0's pod list,
                                 worker has nothing to act on, sits to timeout

Single agent answers the same question in under 80s.

The failure-propagation half, same benchmark (multi-category-findings, gpt-5.2):

  T0  ok
  T1  FAIL  timeout
  T2  blocked            (direct dependent of T1)
  T3  pending forever    never started, silently dropped from the final answer

Evidence

Scanning all stored SRE hard runs with aura-stalls (83 result directories, 9+ models): 92 affected task instances. 3 hard stalls (zero tool calls then timeout), 85 planned tasks dropped after an upstream failure, and 4 tasks re-fetching data an earlier task already had with identical arguments. Another 56 tasks succeeded but ran 5x+ slower than their plan siblings while re-deriving the missing context. That's 12% of all worker task time corpus-wide and 15% in canonical runs. In the run above, the starved task is 92% of the prompt's task time.

Classifying the affected plans: 86 of 90 non-infra cases were valid decompositions, mostly forced by tool contracts (you can't k8s_get_workload before listing workloads) or synthesis steps that combine several earlier results. The plans ask for transitive context, the runtime doesn't deliver it.

Notes

  • SSE events don't include task dependency lists, so none of this is visible from streamed data. Affected runs just look like slow models.
  • Only observed in orchestration mode with sequential plans of 3+ steps. Parallel fan-out is unaffected, the synthesis step directly depends on every branch.
  • A parallel plan dodges the simple case (T0/T1 above are independent, models that fan out rarely hit this). Chained steps like list -> get -> correlate have no plan shape that delivers both earlier results though, so we can't just prompt our way out.
  • Scan is reproducible: uv run aura-stalls sre-hard-results-* in ai-experiments aura-e2e on mshearer/aura-e2e-tests.

Activity

  1. self-assigned this
    on Jun 12, 2026
  2. 23 remaining items

  3. Shearerbeard commented on Jul 17, 2026

    @Shearerbeard
    CollaboratorAuthor

    This work was deemed not the right fix by @teriyakichild and while it improves our scores it has a bit of a smell. Testing and vetting a different approach as a prototype here #374 - must follow up with a ticket.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions