Fine grained controls over collected OTEL metrics

[ingentismg]

Description

In addition to the broader, currently available resource categories (pod, container, node, volume, daemonset, deployment, statefulset and job), we would like to be able to enable and disable any of the available OTEL metrics.
https://opentelemetry.io/docs/specs/semconv/resource/k8s/
https://opentelemetry.io/docs/specs/semconv/system/k8s-metrics/
https://opentelemetry.io/docs/specs/semconv/system/container-metrics/

And other available data.

Reasons

Even enabling a certain category does currently not enable all of the corresponding, available metrics.
On the other hand, not every metric is important for every workload.

In addition to that, certain monitoring systems (e.g. OpenSearch/SAP Cloud Logging) perform very poorly when it comes to efficiently storing metric data.
Being able to fine tune the collected data allows us to minimize the volume of data while having access to metrics that are important to the operation of our workload.