Http Headers are shown as custom labels when open telemetry agent is used

[sjrules]

Question

Hi ,

When I use open telemetry agent in NET 6 web application to send the trace to data Elastic Server , only few of http headers are shown in metadata of trace , and if any additional header is added using activity.SetTag for the http request , http header are shown as custom labels like labels.http_request_headers , but it Elastic agent is used for the same app all http headers are shown under http section and not as custom labels .

Is there any way to show http header under http section instead as custom labels ?

[vishweshbankwar]

@sjrules - It looks like your issue is specific to elastic and not related to OpenTelemetry API/SDK? Could you please clarify?

@stevejgordon - Would you be able to help here?

[stevejgordon]

Thanks for pinging me, @vishweshbankwar. @sjrules This is an artefact of how we transform OTel data to store it into Elastic APM. Any unmapped attributes are currently stored as labels. We are working on improving this, but in this case, I think we should be able to map this data to our Elastic format for storage and a better experience. We have an OTel distro where we can make this happen very quickly, and then we can review fixing the intact mappings as well.

[stevejgordon]

@sjrules Actually, I may have spoken a bit too soon. Those attributes are not semantic conventions, as far as I can see. What instrumentation is producing them? Our mappings will handle transform attributes from semantic conventions to the Elastic data format.

[sjrules]

@stevejgordon Thanks for the reply . I am using Open Telemetry Agent for instrumenting AspNet Application . And I was trying to add http headers under http section of elastic metadata .

Right now I am just using current activity to SetTag for the http request/response , but this does not seem to be working and headers seem to be coming in custom label section of elastic .

[stevejgordon]

@sjrules On the Elastic backend, we receive OTLP data and map/transform it to the Elastic APM data model. We have a bunch of mappings for the semantic convention attributes, etc. Any custom attributes will be mapped under labels.* or numeric_labels.*. So, what you're seeing is the expected behaviour for unmapped attributes. http.request.method should be mapped as it's in the latest semantic conventions. What Elastic APM version are you on?

Headers are a little more complex, but we should look to map the http.request.header.<key> attributes into our model.

[stevejgordon]

@sjrules I discussed this internally on my return from vacation, and we agree that http.request.header.<key> and http.response.header.<key> attributes should be mapped to our data model. I've opened an issue to track this.

[stevejgordon]

@vishweshbankwar I think we can probably close this issue here since it's something we need to solve on the Elastic end and not an issue with the OTel SDK.

[Kielek]

Closing per @stevejgordon comment.