-
Notifications
You must be signed in to change notification settings - Fork 528
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[apmbench] Add trace only benchmarks #15434
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This pull request does not have a backport label. Could you fix it @carsonip? 🙏
|
|
mergify
bot
added
the
backport-8.x
Automated backport to the 8.x branch with mergify
label
Jan 28, 2025
12 tasks
carsonip
added
backport-skip
Skip notification from the automated backport with mergify
and removed
backport-8.x
Automated backport to the 8.x branch with mergify
labels
Feb 3, 2025
endorama
approved these changes
Feb 3, 2025
kruskall
approved these changes
Feb 3, 2025
@Mergifyio backport 9.0 8.x 8.18 |
✅ Backports have been created
|
mergify bot
pushed a commit
that referenced
this pull request
Feb 7, 2025
* Add BenchmarkTracesAgentNodeJS * Add more agents * Update comment * Exclude BenchmarkTraces* from scheduled runs to avoid skewing nightly results * Use loadgen.NewEventHandler (cherry picked from commit 9ad3be4)
2 tasks
mergify bot
pushed a commit
that referenced
this pull request
Feb 7, 2025
* Add BenchmarkTracesAgentNodeJS * Add more agents * Update comment * Exclude BenchmarkTraces* from scheduled runs to avoid skewing nightly results * Use loadgen.NewEventHandler (cherry picked from commit 9ad3be4)
2 tasks
mergify bot
pushed a commit
that referenced
this pull request
Feb 7, 2025
* Add BenchmarkTracesAgentNodeJS * Add more agents * Update comment * Exclude BenchmarkTraces* from scheduled runs to avoid skewing nightly results * Use loadgen.NewEventHandler (cherry picked from commit 9ad3be4)
2 tasks
mergify bot
added a commit
that referenced
this pull request
Feb 7, 2025
* Add BenchmarkTracesAgentNodeJS * Add more agents * Update comment * Exclude BenchmarkTraces* from scheduled runs to avoid skewing nightly results * Use loadgen.NewEventHandler (cherry picked from commit 9ad3be4) Co-authored-by: Carson Ip <[email protected]>
mergify bot
added a commit
that referenced
this pull request
Feb 7, 2025
* Add BenchmarkTracesAgentNodeJS * Add more agents * Update comment * Exclude BenchmarkTraces* from scheduled runs to avoid skewing nightly results * Use loadgen.NewEventHandler (cherry picked from commit 9ad3be4) Co-authored-by: Carson Ip <[email protected]>
mergify bot
added a commit
that referenced
this pull request
Feb 7, 2025
* Add BenchmarkTracesAgentNodeJS * Add more agents * Update comment * Exclude BenchmarkTraces* from scheduled runs to avoid skewing nightly results * Use loadgen.NewEventHandler (cherry picked from commit 9ad3be4) Co-authored-by: Carson Ip <[email protected]> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation/summary
Existing benchmarks are a mix of logs, metrics and traces (sampled and unsampled transactions). Using these benchmarks will introduce noise in benchmark results to assess TBS performance. It will create strange results where the slower the disk, the higher the throughput, because traces ingestion slows and ingestion of other events speeds up to take up the CPU time spared by slow disk, making benchmark results very hard to interpret.
Create a special benchmark with only sampled traces to accurately benchmark traces. Benchmark files are obtained by e.g.
grep -E '^{"span|^{"transaction|^{"meta' apm-nodejs-3.29.0.ndjson > apm-nodejs-traces-3.29.0.ndjson
We may want to revert this in the future to add these files to upstream apm-perf and bump apm-server to use upstream apm-perf e.g. #15379 . It is currently using a very old apm-perf.
Checklist
For functional changes, consider:
How to test these changes
Related issues
Useful to benchmark #15235 and related TBS optimizations