Skip to content

bug(otel): span parenting broken after axum migration #110

Description

@teriyakichild

Summary

After the actix-web → axum migration (cb87687), OTel span parenting breaks in orchestration mode. Two distinct symptoms observed:

  1. mcp.tool_call spans orphaned — created as root spans in separate traces instead of nesting under execute_tool
  2. agent.turn spans intermittently missing from traces

Both symptoms are absent on a backup of the pre-axum branch (backup/artifact-infra-local, commit 65e14cc) with identical configs, confirmed via side-by-side testing.

Expected hierarchy

agent.stream → orchestration → orchestration.worker → agent.turn → execute_tool → mcp.tool_call

Actual behavior

  • mcp.tool_call appears with parentId: null and a different traceId from the orchestration trace
  • execute_tool spans exist in the orchestration trace but have no mcp.tool_call children
  • Intermittent: some traces correctly nest mcp.tool_call, most don't
  • agent.turn occasionally absent from traces

Likely area

Span propagation across tokio::spawn in WrappedTool::call() (crates/aura/src/tool_wrapper.rs:437-441). The Span::current() capture and instrument() propagation worked under actix-web's current_thread runtime but behaves differently with axum's multi-threaded runtime.

Related

Activity

  1. justintime4tea commented on May 15, 2026

    @justintime4tea
    Collaborator

    Current code on main seems to produce the expected results - can we try reproducing this issue with latest main and confirm whether or not this issue still exists in part or in whole?

    Image
  2. self-assigned this
    on Aug 18, 2026
  3. dhable commented on Aug 18, 2026

    @dhable
    Contributor

    Ran through the math_mcp test using commit 474556e. Asking for a sum of the even numbers from 1 to 20 resulted seems to have resulted in a correct trace.

    Image
    • There are mcp.tool_call spans as children of execute_tool spans.
    • agent_turn spans are all present
    • The mcp.tool_call spans all have tool_name=sum

    I did discover with claude-sonnet-4-6 that the model wanted to directly reply with the answer to my simple math queries and skip the MCP call. In order to exercise the MCP tool, I had to set allow_direct_answers = false in the aura config. I don't think there is a bug in the current commit on main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions