Add labels to nodes which indicate the worker pool owner + HA support enabled

[tobiscr]

Description

To make the worker pool feature more transparent and flexible, we are going to add more metadata (as labels) to worker pools. These label have to be added to all nodes.

Indicator for worker-pool owner:
To distinguish the Kyma owned worker pool (name: cpu-worker-0) from customer worker pools, a dedicated label has to be added which indicates whether the worker pool is owned by the customer or by Kyma.

Proposal: worker.kyma.io/pool-owner: [customer|kyma]

Indicator whether worker-pool is running in HA mode:
It should be possible to see if a node belongs to a worker-pool which runs in HA mode. HA is indicated by using 3 different zones for the worker pool.

Proposal: worker.kyma.io/pool-ha-enabled: true/false

AC:

• Define the labels name and value and share it within the team and document it in this issue.
• Adjust KIM to set the labels-field in the Shoot spec:
  • Label indicating whether the worker-pool runs in HA mode
  • Label indiacting the worker-pool owner
    • Reply on comment Configurable multiple node pools [EPIC] kyma#18709 (comment): inform @ingentismg about the feature finalisation and rollout date
    • Update customer documentation how workloads can be assigned to nodes / worker pools (only relevant for pool-owner label).
• Implement an unit-test which confirms the existence of these labels within the Shoot-Spec.

Reasons

Introduce labels to provide metadata related to worker pools. These labels can be used for assigning workloads to the Kyma worker pool, respectively to distinguish the Kyma worker pool from customer worker pools. But also to see if a worker-pool is configured to run in HA mode.

Attachments

[tobiscr]

Required by kyma-project/busola#3621

[mrCherry97]

hi @tobiscr
could we also with this issue add information about the min/max to the labels, or somewhere where Kyma Dashboard can take this data and display in the node details?
MIN-value = amount of used zones in a worker pool (a worker-pool has to have 1 VM per zone), MAX-value=TBC if we can show it - not available in SKR at the moment

[tobiscr]

Yes, from technical side, it's possible to add the autoscaler-configuration as additional label (e.g. as example, we could add one label for min-value worker.kyma.io/pool-autoscaler-min: 3 and an additional label including the max-value worker.kyma.io/pool-autoscaler-max: 20).

What I see a bit critical is the growing number of labels.

But maybe it's worth to expose parts of the Runtime-Configuration in general on SKRs in a CRD. @akgalwas - what do you think about this idea (e.g. we also have a need to expose the public-IP to customers, this would require one more label etc.)?

[mrCherry97]

okay, so please also add it, and we will adjust these informations in node details as follow-up. thanks!

[tobiscr]

Relates to #618