[DOC] Recommendation when using a lot of metrics filters #61
Conversation
Following a report from a user, we discovered it might be necessary to increase `Xss` in `jvm.options`. The (third party) `Metriks::ExponentiallyDecayingSample` uses a red-black tree to ensure the worst-case cost to look up an individual entry is `O(log N)`. It keeps a maximum of 1024 entries around, which means that the worst-case lookup is 10 steps.
|
Hello @karenzone |
| IMPORTANT: When using several `metrics` filters (in particular the `timer`), it might | ||
| become necessary to increase the `Xss` setting in `jvm.options` to avoid `StackOverflowException`. |
There was a problem hiding this comment.
| IMPORTANT: When using several `metrics` filters (in particular the `timer`), it might | |
| become necessary to increase the `Xss` setting in `jvm.options` to avoid `StackOverflowException`. | |
| TIP: If you see `StackOverflowException` erors and are using several `metrics` filters (the `timer` in particular ), try increasing the `Xss` setting in `jvm.options`. |
There was a problem hiding this comment.
Consider rewording as a remedy IF a user sees this problem rather than tempting people to fiddle with jvm settings without need?
There was a problem hiding this comment.
It's not clear to me why the StackOverflowException happens more if they use the particular timer setting. The StackOverflowException is a general problem, it's related to the way Logstash plans the execution of a pipeline.
Explaining better what I mean: suppose the pipeline is configured with 2 filters, I -> F1 - F2 -> O, the execution plan is:
- execute F1
- it's F1 that invokes the execution of F2, so it's like
F2(F1())(NB this is formally incorrect, but it's useful to explain the nesting of the calls)
Every filter count as a stack frame in the execution thread's stack. So this comment is more general in Logstash execution context, it's not specific to this plugin; the tip is part of the general Logstash documentation, where it's already discussed https://www.elastic.co/guide/en/logstash/current/jvm-settings.html#stacks-size
3 participants
Following a report from a user, we discovered it might be necessary to increase
Xssinjvm.options.The (third party)
Metriks::ExponentiallyDecayingSampleuses a red-black tree to ensure the worst-case cost to look up an individual entry isO(log N). It keeps a maximum of 1024 entries around, which means that the worst-case lookup is 10 steps.