monitoring: apm-server not shiping tbs monitoring metrics

[endorama]

Follow up on #13475

Not all metrics have been included in the previous fixes. One of which is beat.stats.apm_server.sampling.tail.*.

Tasks

Preview Give feedback

1. Add missing apm-server tail sampling monitoring metrics to stack monitoring mapping elasticsearch#121543
   :Data Management/Monitoring >enhancement Team:Data Management auto-backport v8.16.4 v8.17.2 v8.18.0 v8.19.0 v9.0.0 v9.1.0 +10
2. metricbeat: Update beat module with apm-server tail sampling monitoring metrics fields beats#42569
   backport-8.16 backport-8.17 backport-8.18 backport-8.x backport-9.0 bug needs_team +7
   
3. [elastic_agent] Add missing apm-server tail sampling monitoring metrics mappings integrations#12568
   Integration:elastic_agent bugfix +2

How to test these PRs

See #13475

Enable TBS, see that metrics are indexed.

[lucabelluccini]

If possible, let's also publish:

• the value of sampling.tail.storage_limit as a number so we can easily plot it
• a boolean metric to tell if TBS is "bypassed" because of the sampling.tail.storage_limit is exceeded (instead of relying on logs)

[carsonip]

the value of sampling.tail.storage_limit as a number so we can easily plot it

Created #15533 to track this

a boolean metric to tell if TBS is "bypassed" because of the sampling.tail.storage_limit is exceeded (instead of relying on logs)

It doesn't have to be a boolean metric. I believe the existing sampling.tail.events.failed_writes should be good enough to determine this. It should be available once this issue is resolved.

[carsonip]

Closing as all PRs and backport PRs are merged. This will need a changelog line in 8.16.4 and 8.17.2: Fix unmapped or dropped APM Server tail sampling monitoring metrics in APM Server self-monitoring, Stack monitoring, Metricbeat, and Elastic Agent integration {issue}14247[14247]

[endorama]

Changelog PR up: #15561