fix(tracing): fix orphan spans in OtelTracingMiddleware by reading parent OTel Context from Reactor ContextView#1940
Open
Buktal wants to merge 1 commit into
Conversation
…rent OTel Context from Reactor ContextView
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Contributor
|
Good fix! Reading parent OTel Context from Reactor ContextView instead of ThreadLocal is the correct approach for Reactor pipelines. The ContextPropagationOperator stores it there for exactly this reason. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AgentScope-Java Version
2.0.0-SNAPSHOT
Description
Background
OtelTracingMiddlewarewas producing orphan spans:chat <model>andexecute_tool <name>spans appeared as disconnected root spans in OTLP backends (e.g. Langfuse) instead of being nested underinvoke_agent. Additionally, anySpanProcessorthat reads the parentContextinonStart(e.g. to inject business metadata) received an empty context.Closes #1938
Root cause
All three hooks (
onAgent,onModelCall,onActing) usedFlux.defer+Context.current()(ThreadLocal) to resolve the parent OTel Context. Inside a Reactor pipeline, the parent span is stored in Reactor Context (written byContextPropagationOperator.runWithContext), not in ThreadLocal. After any thread hop (publishOn/subscribeOn),Context.current()returns an empty context, so child spans had no parent.Fix
Switch all three hooks from
Flux.defertoFlux.deferContextual, and resolve the parent context viaContextPropagationOperator.getOpenTelemetryContextFromContextView(ctxView, Context.current())— this reads from Reactor Context first and falls back to ThreadLocal. Then callspanBuilder.setParent(parentContext)explicitly and build the child context withspan.storeInContext(parentContext).This is the same pattern used in the now-deprecated
TelemetryTracer.callModel.Additional improvements
onActing: replacedLinkedHashSetwithConcurrentHashMap.newKeySet()forcallIds(defensive thread safety)onActing: span name now uses<firstTool> (+N more)for batch tool calls to cap cardinality; full tool names are preserved in thegen_ai.tool.nameattributeresolveOtelContextandbuildToolSpanNamehelpers with clarifying commentsHow to test
Configure any OTLP backend (e.g. Langfuse). Add
OtelTracingMiddlewareto aReActAgent. Invoke the agent and verify:chat <model>andexecute_tool <name>spans appear nested underinvoke_agenttraceIdSpanProcessorthat reads parent context inonStartcorrectly receives business metadataChecklist
mvn spotless:applymvn test)