TBS automatic migration

[raultorrecilla]

Context

Tail Based Sampling (TBS) has been a GA feature in APM Server since major version 8. During this time several SDHs were raised with bugs and issues related to TBS, ranging from high resource usage when TBS is enabled (CPU and memory) to high disk usage and other bugs. The current TBS implementation is backed by badger. We are using v2 which is quite outdated; but the newer version v4 introduces significant changes, especially in increased resource usage. See more details on upgrading badger in #11546.

Based on these issues, we decided to explore replacing badger with pebble, as described in #15246. Over the past weeks significant progress was made with this, and as shared in #15235 (comment), the switch to pebble brings huge performance improvements and decrease in disk space usage.

Hence, we decided to move forward with switching to pebble and the changes are merged into 9.0.

Problem to solve

Achieve an automatic migration from badger to pebble without data loss. Implement it in 8.18 so we have a path forward for customers migrating from 8.18 or upper to 9.0

Special considerations

There might be many special and edge cases, we need to be thorough in the testing phase.

[carsonip]

Closing as we do not plan a migration.

#15235 is going to be a breaking change, and we'll provide public docs to help on-prem customers migrate to 9.0 in #15546. #15235 is not going to be in 8.x and as such there is no need for "automatic migration".