-
Notifications
You must be signed in to change notification settings - Fork 178
Traceparent not forwarded via PubSub #355
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
This code does not forward the tradeid as well
using WithTraceID like
|
It seems that if you pass the grpc trace header with the context, it works then. Not sure if this is intended, or a bug, but as a workaround the following works. you will need either the full import of Then before .PublishEvent, sc := trace.SpanContextFromContext(ctx)
tid := utils.BinaryFromSpanContext(sc)
md := metadata.Pairs("grpc-trace-bin", string(tid))
ctx = metadata.NewOutgoingContext(ctx, md) Then using the new context with grpc metadata appears to propagate the traceparent correctly across dapr, and traces get connected. This is specifically for GRPC client. A HTTP client (I'm not sure if the sdk provides one, I tested simply using http requests to a dapr sidecar) appears to work as-is. |
@linas-ipxo i have done this, and play a little bit with trace, span and ctx it self, but nothing works... at the end i always got "traceid":"00-1234567890abcdef1234567890abcdef-1fa19ef014f128c8-01". maybe you have working example? |
something was achieved using PublishEventWithMetadata , and it looks like this:
more info here: https://docs.dapr.io/developing-applications/building-blocks/pubsub/pubsub-cloudevents/#replace-dapr-generated-cloudevents-values |
I'm not entirely sure what you mean, or expected to see, but the following code is what I was using while testing this. package daprtest
import (
"context"
"encoding/json"
"go-api/services/trace"
dapr "github.com/dapr/go-sdk/client"
otel "go.opentelemetry.io/otel"
sdktrace "go.opentelemetry.io/otel/trace"
"google.golang.org/grpc/metadata"
"github.com/google/uuid"
cloudevents "github.com/cloudevents/sdk-go/v2"
)
var daprClient dapr.Client
func DoTheThing() {
var err error
daprClient, err = dapr.NewClient()
if err != nil {
panic("cannot create dapr client")
}
// context would come from some other place, which has instantiated the parent span already
// otherwise this will be the root span with new trace id
ctx := context.Background()
_, span := trace.Tracer.Start(ctx, "lalalal")
defer span.End()
data := map[string]string{"something": "happened"}
publishEvent("topic", ctx, data)
}
func publishEvent(topic string, ctx context.Context, data interface{}) {
sc := sdktrace.SpanContextFromContext(ctx)
tid := string(binaryFromSpanContext(sc))
md := metadata.Pairs("grpc-trace-bin", tid)
newCtx := metadata.NewOutgoingContext(ctx, md)
err := daprClient.PublishEvent(
newCtx, // this new context contains grpc metadata with "grpc-trace-bin"
"pubsub",
topic,
formatRawCloudEvent(topic, ctx, data),
)
if err != nil {
panic(err)
}
}
// https://github.com/dapr/dapr/blob/release-1.13/pkg/diagnostics/utils/trace_utils.go#L134
func binaryFromSpanContext(sc sdktrace.SpanContext) []byte {
traceID := sc.TraceID()
spanID := sc.SpanID()
traceFlags := sc.TraceFlags()
var b [29]byte
copy(b[2:18], traceID[:])
b[18] = 1
copy(b[19:27], spanID[:])
b[27] = 2
b[28] = uint8(traceFlags)
return b[:]
}
func formatRawCloudEvent(topic string, ctx context.Context, data interface{}) string {
event := cloudevents.NewEvent()
event.SetID(uuid.NewString())
event.SetSource("test-source")
event.SetType(topic)
carrier := CloudEventCarrier{Event: &event}
event.SetData("application/json", data)
otel.GetTextMapPropagator().Inject(ctx, carrier)
j, err := json.Marshal(event)
if err != nil {
panic(err)
}
return string(j)
}
/** cloud event carrier for otel */
type CloudEventCarrier struct {
Event *cloudevents.Event
}
// Get returns the value associated with the passed key.
func (c CloudEventCarrier) Get(key string) string {
ext, err := c.Event.Context.GetExtension(key)
if err != nil {
return ""
}
return ext.(string)
}
// Set stores the key-value pair.
func (c CloudEventCarrier) Set(key, value string) {
c.Event.Context.SetExtension(key, value)
}
// Keys lists the keys stored in this carrier.
func (c CloudEventCarrier) Keys() []string {
ext := c.Event.Context.GetExtensions()
keys := make([]string, 0, len(ext))
for k := range ext {
keys = append(keys, k)
}
return keys
} |
Uh oh!
There was an error while loading. Please reload this page.
I don't know if this is a bug or feature request. I have two services "server" and "echo". Server sends send a message via pubsub to echo. Both services are fully instrumented with OpenTelemetry. A call to server give me the following trace
The last method SendNotifcation forwards the instrumented ctx to the DAPR client:
But the transported message has a new traceid:
Receiving the trace works again:
But alltogether I have two traces:
Questions:
The text was updated successfully, but these errors were encountered: