[apmbench] Add trace only benchmarks (backport #15434) #15603
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![mergify[bot]](https://avatars.githubusercontent.com/in/10562?s=60&v=4){kind=small}
* Add BenchmarkTracesAgentNodeJS * Add more agents * Update comment * Exclude BenchmarkTraces* from scheduled runs to avoid skewing nightly results * Use loadgen.NewEventHandler (cherry picked from commit 9ad3be4)
2 tasks
carsonip
approved these changes
Feb 7, 2025
✅ The pull request has been merged automaticallyThe pull request has been merged automatically at 7665a4d |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation/summary
Existing benchmarks are a mix of logs, metrics and traces (sampled and unsampled transactions). Using these benchmarks will introduce noise in benchmark results to assess TBS performance. It will create strange results where the slower the disk, the higher the throughput, because traces ingestion slows and ingestion of other events speeds up to take up the CPU time spared by slow disk, making benchmark results very hard to interpret.
Create a special benchmark with only sampled traces to accurately benchmark traces. Benchmark files are obtained by e.g.
grep -E '^{"span|^{"transaction|^{"meta' apm-nodejs-3.29.0.ndjson > apm-nodejs-traces-3.29.0.ndjsonWe may want to revert this in the future to add these files to upstream apm-perf and bump apm-server to use upstream apm-perf e.g. #15379 . It is currently using a very old apm-perf.
Checklist
For functional changes, consider:
How to test these changes
Related issues
Useful to benchmark #15235 and related TBS optimizations
This is an automatic backport of pull request #15434 done by [Mergify](https://mergify.com).