refactor(model-serving): remove controller label propagation and add autoscaling example

[WHOIM1205]

Summary

This PR updates the autoscaling example for ModelServing based on the feedback received.

Instead of adding any controller-level logic or fixed labels, the example now relies on user-defined labels in the ModelServing templates.

---

What was removed

• Removed controller-side label propagation logic
• Removed the fixed annotation (kthena.io/model-name)
• Removed related constants and tests

The idea here is to keep the controller generic and not enforce any specific labeling pattern. Users can define labels as needed.

---

What was added

• Added an autoscaling example:
  examples/model-serving/autoscaling-with-keda.yaml

The example demonstrates:

• Defining labels in ModelServing templates
• Using Prometheus metrics for scaling
• Configuring KEDA to scale ModelServing

---

Fixes & Improvements

• Fixed Prometheus query label:

  • model_name → model (to match the actual metric)

• Replaced hardcoded values with placeholders:

  • <model-name>
  • <modelserving-name>
  • <prometheus-url>

• Clarified a few things in comments:

  • Metrics come from the router, not from pod labels
  • Labels are optional and user-defined
  • These are standard pod-template labels (nothing special added in the controller)

• Added:

  • pollingInterval: 15
  • cooldownPeriod: 120 (can be tuned depending on workload)

---

How it works

1. User defines labels in the ModelServing template
2. These labels are part of the pod template and show up on the created pods
3. The router emits metrics with a model label
4. KEDA queries Prometheus using that label
5. HPA updates spec.replicas on ModelServing

---

Notes

• No changes to controller behavior
• No CRD changes
• Fully backward compatible
• Autoscaling is optional and user-driven

---

Context

In an earlier iteration of this PR, I tried adding controller-side label propagation, but based on feedback, that approach was dropped.

The current version keeps things simple and just adds an example using user-defined labels.

---

Goal

Provide a simple and flexible example of autoscaling with Prometheus + KEDA, without introducing any opinionated controller changes.

[gemini-code-assist]

Code Review

This pull request introduces the propagation of the 'kthena.io/model-name' annotation from the ModelServing CR to the corresponding pod labels. The implementation includes validation to ensure the annotation value is a valid Kubernetes label and adds comprehensive unit tests to verify the propagation logic and precedence rules. I have reviewed the code and suggest removing the redundant '!exists' check in the label assignment logic, as the pod labels are initialized with a fixed set of keys that do not include this annotation.

[gemini-code-assist]

The !exists check is redundant here because pod.Labels was just initialized a few lines above (lines 135-142) with a fixed set of keys that does not include ModelNameAnnotationKey. Removing this check simplifies the logic without changing behavior, as any subsequent overrides from the role template are handled later in addPodLabelAndAnnotation.

else {
			pod.Labels[workloadv1alpha1.ModelNameAnnotationKey] = modelName
		}

[FAUST-BENCHOU]

/retest

[volcano-sh-bot]

@FAUST-BENCHOU: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

Details

In response to this:

/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[volcano-sh-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign yaozengzeng for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• examples/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[FAUST-BENCHOU]

dont see any remove work, only a yml example

[WHOIM1205]

dont see any remove work, only a yml example

Hey @FAUST-BENCHOU thanks for the feedback

ive cleaned up the pr a bit removed the earlier controller side label propagation approach and kept it aligned with using user defined labels only also squashed the commits so the diff reflects the final state more clearly

right now it just adds the autoscaling example with prometheus and keda without introducing any controller changes

happy to adjust anything further if needed

[LiZhenCheng9527]

dont see any remove work, only a yml example

Hey @FAUST-BENCHOU thanks for the feedback

ive cleaned up the pr a bit removed the earlier controller side label propagation approach and kept it aligned with using user defined labels only also squashed the commits so the diff reflects the final state more clearly

right now it just adds the autoscaling example with prometheus and keda without introducing any controller changes

happy to adjust anything further if needed

If you’ve done these things, you should update the PR title and description.