Add terraform module to benchmark Elasticsearch on Elastic Cloud with locally built APM-Server and Integration package #8986

lahsivjar · 2022-08-26T15:19:03Z

Motivation/summary

Allows running rally to benchmark a locally built Elastic Cloud cluster. This allows developers to benchmark changes introduced in ES schema used by APM-Server.

Checklist

~~- [ ] Update CHANGELOG.asciidoc~~
~~- [ ] Update package changelog.yml (only if changes to apmpackage have been made)~~
~~- [ ] Documentation has been updated~~

How to test these changes

Navigate to testing/rally-cloud and run make init if you are running for the first
time
Get EC_API_KEY by following this guide.
Checkout the testing/rally-cloud/variables.tf to customize your benchmark.
Run EC_API_KEY=<ec_api_key> make apply to create the required infra and run rally.

The rally benchmark will execute everytime make apply is called however, the infrastucture
will be created only once. After testing is done make sure to destroy all infrastructure using
EC_API_KEY=<ec_api_key> make destroy

Related issues

Closes #8754

mergify · 2022-08-26T15:22:42Z

This pull request does not have a backport label. Could you fix it @lahsivjar? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

backport-7.x is the label to automatically backport to the 7.x branch.
backport-7./d is the label to automatically backport to the 7./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

apmmachine · 2022-08-26T15:25:57Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2022-09-06T07:53:11.356+0000
Duration: 25 min 53 sec

Test stats 🧪

Test	Results
Failed	0
Passed	130
Skipped	0
Total	130

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate and publish the docker images.
/test windows : Build & tests on Windows.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

apmmachine · 2022-08-26T16:08:29Z

📚 Go benchmark report

Diff with the main branch

name                                                                                              old time/op    new time/op     delta
pkg:github.com/elastic/apm-server/internal/agentcfg goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/internal/beater/request goos:linux goarch:amd64
ContextReset/Forwarded_ipv6-12                                                                       169ns ± 1%      167ns ± 0%   -1.52%  (p=0.008 n=5+5)
ContextReset/Remote_Addr_ipv4-12                                                                    81.5ns ± 0%     80.8ns ± 1%   -0.83%  (p=0.016 n=5+5)
ContextReset/Remote_Addr_ipv6-12                                                                     110ns ± 2%      108ns ± 0%   -1.86%  (p=0.008 n=5+5)
pkg:github.com/elastic/apm-server/internal/decoder goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/internal/model/modelindexer goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/internal/processor/stream goos:linux goarch:amd64
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/metadata.ndjson-12                      4.42µs ± 4%     4.75µs ±10%   +7.47%  (p=0.032 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel4/metadata.ndjson-12                      2.73µs ± 4%     3.12µs ± 4%  +14.11%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel4/optional-timestamps.ndjson-12           3.44µs ± 6%     3.94µs ± 1%  +14.43%  (p=0.016 n=5+4)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/events.ndjson-12                        15.6µs ± 1%     15.8µs ± 1%   +1.05%  (p=0.032 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/invalid-json-event.ndjson-12            1.40µs ± 1%     1.41µs ± 1%   +1.13%  (p=0.048 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/ratelimit.ndjson-12                     6.70µs ± 2%     7.14µs ± 1%   +6.57%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/span-links.ndjson-12                    1.51µs ± 1%     1.58µs ± 2%   +4.99%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/spans.ndjson-12                         10.8µs ± 2%     11.4µs ± 0%   +5.53%  (p=0.016 n=5+4)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/transactions-huge_traces.ndjson-12      5.46µs ± 1%     5.65µs ± 2%   +3.51%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/transactions.ndjson-12                  10.8µs ± 1%     11.4µs ± 1%   +5.54%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/transactions_spans.ndjson-12            10.6µs ± 2%     11.3µs ± 2%   +6.60%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/transactions_spans_rum.ndjson-12        1.95µs ± 2%     2.06µs ± 1%   +5.78%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/transactions_spans_rum_2.ndjson-12      1.88µs ± 1%     1.97µs ± 1%   +5.08%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/unknown-span-type.ndjson-12             7.14µs ± 2%     7.46µs ± 1%   +4.48%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/errors.ndjson-12                      6.82µs ± 2%     7.60µs ± 3%  +11.45%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/errors_2.ndjson-12                    6.56µs ± 5%     7.07µs ± 1%   +7.72%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/errors_rum.ndjson-12                  1.95µs ± 3%     2.01µs ± 1%   +3.41%  (p=0.016 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/errors_transaction_id.ndjson-12       5.35µs ± 3%     5.54µs ± 1%   +3.55%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/events.ndjson-12                      13.3µs ± 1%     14.0µs ± 1%   +4.97%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/heavy.ndjson-12                        544µs ± 2%      567µs ± 2%   +4.28%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/invalid-event-type.ndjson-12           767ns ± 1%      800ns ± 0%   +4.26%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/invalid-event.ndjson-12               3.28µs ± 2%     3.41µs ± 1%   +4.11%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/invalid-json-event.ndjson-12          1.05µs ± 2%     1.10µs ± 3%   +4.88%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/invalid-json-metadata.ndjson-12       1.87µs ± 0%     1.95µs ± 0%   +4.53%  (p=0.016 n=5+4)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/metadata.ndjson-12                    1.23µs ± 2%     1.28µs ± 1%   +3.34%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/metricsets.ndjson-12                  4.55µs ± 2%     4.69µs ± 1%   +3.19%  (p=0.016 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/minimal-service.ndjson-12             1.04µs ± 2%     1.09µs ± 3%   +5.01%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/optional-timestamps.ndjson-12         1.53µs ± 1%     1.58µs ± 1%   +3.35%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/otel-bridge.ndjson-12                 3.11µs ± 1%     3.25µs ± 1%   +4.38%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/span-links.ndjson-12                  1.16µs ± 1%     1.21µs ± 2%   +4.33%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/spans.ndjson-12                       9.33µs ± 1%     9.69µs ± 1%   +3.94%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/transactions-huge_traces.ndjson-12    4.55µs ± 2%     4.78µs ± 2%   +5.24%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/transactions.ndjson-12                9.25µs ± 2%     9.63µs ± 3%   +4.11%  (p=0.016 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/transactions_spans.ndjson-12          9.32µs ± 2%     9.62µs ± 2%   +3.22%  (p=0.016 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/transactions_spans_rum.ndjson-12      1.58µs ± 1%     1.63µs ± 2%   +3.15%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/transactions_spans_rum_2.ndjson-12    1.51µs ± 2%     1.57µs ± 1%   +3.86%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/unknown-span-type.ndjson-12           6.04µs ± 2%     6.26µs ± 1%   +3.67%  (p=0.008 n=5+5)
ReadBatch/invalid-metadata-2.ndjson-12                                                              29.3µs ±15%     22.1µs ±24%  -24.53%  (p=0.016 n=5+5)
ReadBatch/minimal-service.ndjson-12                                                                 15.7µs ± 7%     12.6µs ±19%  -19.60%  (p=0.032 n=5+5)
ReadBatch/span-links.ndjson-12                                                                      14.0µs ±23%     17.1µs ±13%  +22.18%  (p=0.032 n=5+5)
pkg:github.com/elastic/apm-server/internal/publish goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/aggregation/spanmetrics goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/aggregation/txmetrics goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/sampling goos:linux goarch:amd64
TraceGroups-12                                                                                       122ns ± 2%      157ns ± 5%  +28.54%  (p=0.008 n=5+5)
pkg:github.com/elastic/apm-server/x-pack/apm-server/sampling/eventstorage goos:linux goarch:amd64
ReadEvents/json_codec_big_tx/0_events-12                                                             337ns ± 9%      309ns ± 6%   -8.35%  (p=0.032 n=5+5)

name                                                                                              old alloc/op   new alloc/op    delta
pkg:github.com/elastic/apm-server/internal/agentcfg goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/internal/beater/request goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/internal/decoder goos:linux goarch:amd64
CompressedRequestReader/gzip_content_encoding-12                                                    45.6kB ± 0%     45.6kB ± 0%   +0.10%  (p=0.024 n=5+5)
pkg:github.com/elastic/apm-server/internal/model/modelindexer goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/internal/processor/stream goos:linux goarch:amd64
RUMV3Processor/rum_events.ndjson-12                                                                 3.05kB ± 1%     3.08kB ± 2%   +1.17%  (p=0.048 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/transactions_spans_rum.ndjson-12        6.16kB ± 1%     6.05kB ± 2%   -1.82%  (p=0.032 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel4/invalid-metadata.ndjson-12              3.22kB ± 1%     3.29kB ± 1%   +1.94%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel4/spans.ndjson-12                         30.8kB ± 0%     30.7kB ± 0%   -0.39%  (p=0.016 n=5+4)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/metadata.ndjson-12                      5.53kB ± 1%     5.47kB ± 0%   -1.20%  (p=0.016 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/minimal-service.ndjson-12               5.27kB ± 0%     5.32kB ± 1%   +1.05%  (p=0.032 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/events.ndjson-12                      43.6kB ± 1%     43.2kB ± 0%   -0.89%  (p=0.016 n=5+5)
ReadBatch/errors_2.ndjson-12                                                                        22.6kB ± 0%     22.6kB ± 0%   -0.06%  (p=0.024 n=5+5)
ReadBatch/heavy.ndjson-12                                                                           1.27MB ± 0%     1.27MB ± 0%   +0.14%  (p=0.008 n=5+5)
pkg:github.com/elastic/apm-server/internal/publish goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/aggregation/spanmetrics goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/aggregation/txmetrics goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/sampling goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/sampling/eventstorage goos:linux goarch:amd64

name                                                                                              old allocs/op  new allocs/op   delta
pkg:github.com/elastic/apm-server/internal/agentcfg goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/internal/beater/request goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/internal/decoder goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/internal/model/modelindexer goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/internal/processor/stream goos:linux goarch:amd64
BackendProcessorParallel/BenchmarkBackendProcessorParallel4/spans.ndjson-12                            573 ± 0%        574 ± 0%   +0.17%  (p=0.029 n=4+4)
ReadBatch/heavy.ndjson-12                                                                            22.6k ± 0%      22.6k ± 0%   +0.01%  (p=0.016 n=5+5)
pkg:github.com/elastic/apm-server/internal/publish goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/aggregation/spanmetrics goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/aggregation/txmetrics goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/sampling goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/sampling/eventstorage goos:linux goarch:amd64

name                                                                                              old speed      new speed       delta
pkg:github.com/elastic/apm-server/internal/processor/stream goos:linux goarch:amd64
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/metadata.ndjson-12                     282MB/s ± 4%    263MB/s ± 9%   -6.70%  (p=0.032 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel4/metadata.ndjson-12                     455MB/s ± 4%    399MB/s ± 4%  -12.29%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel4/optional-timestamps.ndjson-12          299MB/s ± 5%    261MB/s ± 1%  -12.73%  (p=0.016 n=5+4)
BackendProcessorParallel/BenchmarkBackendProcessorParallel4/transactions_spans.ndjson-12           268MB/s ± 1%    288MB/s ± 7%   +7.43%  (p=0.032 n=4+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/events.ndjson-12                       476MB/s ± 1%    471MB/s ± 1%   -1.04%  (p=0.032 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/ratelimit.ndjson-12                    629MB/s ± 2%    590MB/s ± 1%   -6.18%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/span-links.ndjson-12                   452MB/s ± 1%    431MB/s ± 2%   -4.76%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/spans.ndjson-12                        747MB/s ± 2%    708MB/s ± 0%   -5.25%  (p=0.016 n=5+4)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/transactions-huge_traces.ndjson-12     581MB/s ± 1%    561MB/s ± 2%   -3.39%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/transactions.ndjson-12                 523MB/s ± 1%    495MB/s ± 1%   -5.25%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/transactions_spans.ndjson-12           549MB/s ± 2%    515MB/s ± 2%   -6.19%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/transactions_spans_rum.ndjson-12       594MB/s ± 2%    561MB/s ± 1%   -5.46%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/transactions_spans_rum_2.ndjson-12     594MB/s ± 1%    566MB/s ± 1%   -4.83%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/unknown-span-type.ndjson-12            463MB/s ± 2%    443MB/s ± 1%   -4.29%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/errors.ndjson-12                     930MB/s ± 2%    834MB/s ± 3%  -10.27%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/errors_2.ndjson-12                   719MB/s ± 5%    667MB/s ± 1%   -7.26%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/errors_rum.ndjson-12                 975MB/s ± 3%    943MB/s ± 1%   -3.32%  (p=0.016 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/errors_transaction_id.ndjson-12      714MB/s ± 3%    690MB/s ± 1%   -3.43%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/events.ndjson-12                     557MB/s ± 1%    531MB/s ± 1%   -4.73%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/heavy.ndjson-12                      734MB/s ± 2%    704MB/s ± 3%   -4.09%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/invalid-event-type.ndjson-12         510MB/s ± 1%    489MB/s ± 0%   -4.09%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/invalid-event.ndjson-12              234MB/s ± 2%    224MB/s ± 1%   -3.96%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/invalid-json-event.ndjson-12         557MB/s ± 1%    531MB/s ± 3%   -4.65%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/invalid-json-metadata.ndjson-12      239MB/s ± 0%    229MB/s ± 0%   -4.33%  (p=0.016 n=5+4)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/metadata.ndjson-12                  1.01GB/s ± 2%   0.97GB/s ± 1%   -3.21%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/metricsets.ndjson-12                 560MB/s ± 2%    543MB/s ± 1%   -3.12%  (p=0.016 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/minimal-service.ndjson-12            410MB/s ± 1%    390MB/s ± 3%   -4.75%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/optional-timestamps.ndjson-12        672MB/s ± 2%    650MB/s ± 1%   -3.26%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/otel-bridge.ndjson-12                605MB/s ± 1%    579MB/s ± 1%   -4.20%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/span-links.ndjson-12                 586MB/s ± 1%    562MB/s ± 2%   -4.14%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/spans.ndjson-12                      861MB/s ± 1%    829MB/s ± 1%   -3.79%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/transactions-huge_traces.ndjson-12   698MB/s ± 2%    663MB/s ± 2%   -4.98%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/transactions.ndjson-12               611MB/s ± 2%    587MB/s ± 2%   -3.95%  (p=0.016 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/transactions_spans.ndjson-12         625MB/s ± 2%    605MB/s ± 2%   -3.13%  (p=0.016 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/transactions_spans_rum.ndjson-12     730MB/s ± 1%    707MB/s ± 2%   -3.04%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/transactions_spans_rum_2.ndjson-12   741MB/s ± 2%    713MB/s ± 1%   -3.73%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/unknown-span-type.ndjson-12          548MB/s ± 2%    528MB/s ± 1%   -3.54%  (p=0.008 n=5+5)
ReadBatch/invalid-metadata-2.ndjson-12                                                            15.0MB/s ±16%   20.1MB/s ±21%  +33.83%  (p=0.016 n=5+5)
ReadBatch/minimal-service.ndjson-12                                                               27.2MB/s ± 7%   34.3MB/s ±18%  +26.05%  (p=0.032 n=5+5)
ReadBatch/span-links.ndjson-12                                                                    49.4MB/s ±20%   40.0MB/s ±14%  -19.03%  (p=0.032 n=5+5)

report generated with https://pkg.go.dev/golang.org/x/perf/cmd/benchstat

axw

Looks great!

Regarding the need to run make in testing/cloud: perhaps instead of building a custom Kibana image, we would be better off using the default image, and uploading a custom integration package to Kibana after it starts like we do in the system tests:

apm-server/systemtest/kibana.go

Line 137 in fd81083

if err := Fleet.InstallPackageByUpload(f); err != nil {

dev_docs/rally.md

testing/infra/terraform/modules/rally_workers/header.md

testing/rally-cloud/Makefile

testing/rally-cloud/variables.tf

testing/rally-cloud/main.tf

mergify · 2022-09-06T01:23:18Z

This pull request is now in conflicts. Could you fix it @lahsivjar? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 8754_rally-remote upstream/8754_rally-remote
git merge upstream/main
git push upstream 8754_rally-remote

axw

Nice - uploading the custom package is much more sensible than bundling in a custom Kibana. We should probably update testing/cloud later to make use of this too.

It would be great if we could combine the two new vars into one.

testing/infra/terraform/modules/ec_deployment/variables.tf

axw

Looks great :)

lahsivjar · 2022-09-06T07:26:41Z

/test

lahsivjar · 2022-09-06T07:52:56Z

/test

axw · 2022-09-27T06:40:18Z

I tested this earlier. Not with a BC, since it's independent of the stack release.

lahsivjar changed the title ~~8754 rally remote~~ Add terraform module to benchmark Elasticsearch on Elastic Cloud with locally built APM-Server and Integration package Aug 26, 2022

lahsivjar added 2 commits August 26, 2022 23:22

Allow gencorpora to override root dir for source meta

5407efd

Add terraform module to run rally on Elastic cloud

36575cf

lahsivjar force-pushed the 8754_rally-remote branch from 2cc16dc to 36575cf Compare August 26, 2022 15:22

mergify bot added the backport-skip Skip notification from the automated backport with mergify label Aug 26, 2022

lahsivjar requested a review from a team August 26, 2022 15:23

Fix docs

50b5452

axw reviewed Sep 1, 2022

View reviewed changes

lahsivjar added 4 commits September 5, 2022 09:28

Fix typos and remove unncessary variables

6c26329

Auto select gcp_zone from available zones

8dc526d

Remove APM configuration for ec_deployment

53d3a30

Install custom apm integration pkg on default version

52bb5a3

lahsivjar added 3 commits September 6, 2022 09:25

Merge main and fix conflict

7c8c436

Fix typo

3352c05

Remove redundant variables

cffea5a

axw reviewed Sep 6, 2022

View reviewed changes

testing/infra/terraform/modules/ec_deployment/variables.tf Outdated Show resolved Hide resolved

lahsivjar added 4 commits September 6, 2022 12:21

Simplify variables for custom integration pkg

ee70fa9

Fix makefile for gencorpora

5484404

Merge branch 'main' into 8754_rally-remote

2db82ac

Fix rally-cloud readme

f28d274

axw approved these changes Sep 6, 2022

View reviewed changes

Merge branch 'main' into 8754_rally-remote

40c94f8

lahsivjar enabled auto-merge (squash) September 6, 2022 07:28

lahsivjar merged commit 8a951fc into elastic:main Sep 6, 2022

lahsivjar deleted the 8754_rally-remote branch September 6, 2022 08:21

lahsivjar added the v8.5.0 label Sep 27, 2022

axw added test-plan-ok test-plan labels Sep 27, 2022

axw self-assigned this Sep 27, 2022

axw mentioned this pull request Oct 29, 2022

Re-investigate using rally #6115

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add terraform module to benchmark Elasticsearch on Elastic Cloud with locally built APM-Server and Integration package #8986

Add terraform module to benchmark Elasticsearch on Elastic Cloud with locally built APM-Server and Integration package #8986

lahsivjar commented Aug 26, 2022 •

edited

Loading

mergify bot commented Aug 26, 2022

apmmachine commented Aug 26, 2022 •

edited

Loading

Build stats

Test stats 🧪

apmmachine commented Aug 26, 2022 •

edited

Loading

axw left a comment

mergify bot commented Sep 6, 2022

axw left a comment

axw left a comment

lahsivjar commented Sep 6, 2022

lahsivjar commented Sep 6, 2022

axw commented Sep 27, 2022

Add terraform module to benchmark Elasticsearch on Elastic Cloud with locally built APM-Server and Integration package #8986

Add terraform module to benchmark Elasticsearch on Elastic Cloud with locally built APM-Server and Integration package #8986

Conversation

lahsivjar commented Aug 26, 2022 • edited Loading

Motivation/summary

Checklist

How to test these changes

Related issues

mergify bot commented Aug 26, 2022

apmmachine commented Aug 26, 2022 • edited Loading

💚 Build Succeeded

Build stats

Test stats 🧪

🤖 GitHub comments

apmmachine commented Aug 26, 2022 • edited Loading

📚 Go benchmark report

axw left a comment

Choose a reason for hiding this comment

mergify bot commented Sep 6, 2022

axw left a comment

Choose a reason for hiding this comment

axw left a comment

Choose a reason for hiding this comment

lahsivjar commented Sep 6, 2022

lahsivjar commented Sep 6, 2022

axw commented Sep 27, 2022

lahsivjar commented Aug 26, 2022 •

edited

Loading

apmmachine commented Aug 26, 2022 •

edited

Loading

apmmachine commented Aug 26, 2022 •

edited

Loading