HTTP Spans very inaccurate

[jeengbe]

We are seeing vastly different timings and offsets of spans for what should be identical transactions. For some transactions, spans are missing or duplicate.

I'd rather not disclose specifics in this public Discussion, as they include references to private internal systems. GitHub Discussions still looks like the right place to share results and keep track of the status.

Please contact me on Discord (jeengbe#4354) or email [email protected] :)

[jeengbe]

To add some context: We're running a GraphQL server (using graphql-yoga and @envelop/sentry) in production and sampling HTTP spans. The issue now is that with transactions (started by @envelop/sentry), these HTTP spans are very chaotic. On some transactions, certain spans are missing while other transactions contain too many. Also, they are sometimes (following logic in the code) out of order.

[jeengbe]

If I didn't know better I would believe that this is a flaw by design: with this running in production, the server is serving multiple requests simultaneously: request A is busy awaiting an external service, so request B is processed in the meantime. With Sentry's global approach to scopes (const scope = getCurrentHub().getScope();, https://github.com/getsentry/sentry-javascript/blob/develop/packages/node/src/integrations/http.ts#L191) one request might overwrite the scope data of a prior, still executing, query.

[lforst]

Hi, this is something we do consider. getCurrentHub() will pick up the hub in the current domain. If you're using any of our server integrations we take care of this for you by wrapping your request handlers in individual domains to avoid scope bleed.

@envelop/sentry is not maintained by us and if you run into scope bleeding with their package I recommend you raise an issue with them. Feel free to point them to us if they have any questions.

[jeengbe]

That's very interesting!

Does that mean that, if I use the official express middleware, getCurrentHub() returns something different depending on the request context it is run in?

app.use(Sentry.Handlers.requestHandler());

app.get("/A", async (_, res) => {
  await new Promise(r => setTimeout(r, 1000));
  const scope = Sentry.getCurrentHub().getScope();
  res.end("A");
});

app.get("/B", (_, res) => {
  await new Promise(r => setTimeout(r, 5000));
  res.end("B");
});

If I request /A and /B right after each other, is scope still bound to the scope created by the first request, even tough it would seem the second request (to B) would overwrite the currently set scope?

[jeengbe]

If so, that is something I, and apparently authors of 3rd party Sentry extensions, are not aware of and would do very well in some sort of documentation. I didn't know something like that was even possible, thus assumed Sentry-maintained packages would run into the same concurrency issue.

[lforst]

Does that mean that, if I use the official express middleware, getCurrentHub() returns something different depending on the request context it is run in?

Correct.

If so, that is something I, and apparently authors of 3rd party Sentry extensions, are not aware of and would do very well in some sort of documentation. I didn't know something like that was even possible, thus assumed Sentry-maintained packages would run into the same concurrency issue.

That may very well be the case and it's not optimal. We try to abstract as much of this away as possible because it is indeed very edge-casey and can be ugly at times. Ideally, users shouldn't have to worry about concurrency when interacting with the SDK. For integration developers it is obviously different - sadly, our docs do lack information in that regard and we basically always say "do what we do in the SDK". We'll probably not dedicate efforts toward this anytime soon ourselves but PRs are always welcome.

[jeengbe]

Understandable, thank you for the explanation. I'll see if I can find some time.

[jeengbe]

I do not nearly know enough about this to write a doc entry.