KEP: MultiKueue external custom Job support #2458

mszadkow · 2024-06-20T13:58:10Z

What type of PR is this?

/kind documentation

What this PR does / why we need it:

New KEP that explains the way to introduce MultiKueue external custom Job support.

Which issue(s) this PR fixes:

Fixes #2349

Special notes for your reviewer:

Does this PR introduce a user-facing change?

NONE

k8s-ci-robot · 2024-06-20T13:58:15Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: mszadkow
Once this PR has been reviewed and has the lgtm label, please assign alculquicondor for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot · 2024-06-20T13:58:19Z

Hi @mszadkow. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

netlify · 2024-06-20T13:58:58Z

✅ Deploy Preview for kubernetes-sigs-kueue canceled.

Name	Link
🔨 Latest commit	`80ef47d`
🔍 Latest deploy log	https://app.netlify.com/sites/kubernetes-sigs-kueue/deploys/66f3f2eea5ec0c0008aa073d

keps/2349-multikueue-external-custom-job-support/README.md

trasc · 2024-06-25T10:31:56Z

/cc @alculquicondor @tenzen-y

alculquicondor · 2024-06-26T18:24:19Z

/ok-to-test

alculquicondor · 2024-06-26T18:25:39Z

@dgrove-oss is this something that interests you and do you have any thoughts on the proposal?

keps/2349-multikueue-external-custom-job-support/README.md

alculquicondor · 2024-06-26T18:39:17Z

keps/2349-multikueue-external-custom-job-support/README.md

+	}
+}
+```
+


Please expand on:

which subcomponents of MK each external controller is implementing (even if provided by a framework)

how does a configuration for multiple of these controllers look like from the ClusterQueue point of view.

dgrove-oss

The integration test scenario seemed reasonable to me.

Providing a mechanism to register an external controller and have it coordinate with the builtin components about which GVKs each is responsible seems right.

There was not sufficient detail for me as a MultiKueue outsider to understand how the pieces fit together and what I'd need to do to add an external custom Job. Is this KEP also supposed to cover external Job adapters too?

keps/2349-multikueue-external-custom-job-support/README.md

alculquicondor · 2024-06-26T18:51:24Z

Is this KEP also supposed to cover external Job adapters too?

Yes, it's supposed to enable the easy creation of them. I agree that the proposal needs more detail.

tenzen-y · 2024-06-28T13:21:40Z

/cc @alculquicondor @tenzen-y

ACK
Let me check this KEP

alculquicondor · 2024-06-28T14:43:39Z

/retitle [WIP] KEP: MultiKueue external custom Job support

mszadkow · 2024-09-25T11:24:54Z

/retest

k8s-triage-robot · 2024-12-24T12:05:54Z

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

Mark this PR as fresh with /remove-lifecycle stale
Close this PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

dgrove-oss · 2024-12-24T16:53:43Z

/remove-lifecycle stale

k8s-triage-robot · 2025-03-24T17:46:10Z

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

Mark this PR as fresh with /remove-lifecycle stale
Close this PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot · 2025-04-23T18:21:26Z

The Kubernetes project currently lacks enough active contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

Mark this PR as fresh with /remove-lifecycle rotten
Close this PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

yibozhuang · 2025-05-23T00:58:48Z

/remove-lifecycle rotten

k8s-triage-robot · 2025-08-21T01:15:57Z

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

Mark this PR as fresh with /remove-lifecycle stale
Close this PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

gbenhaim · 2025-08-22T12:47:23Z

keps/2349-multikueue-external-custom-job-support/README.md

+## Design Details
+
+Provide a way to integrate external controllers to manage Jobs in the Kueue.
+MulikueueCluster API needs to be modified to ensure that one multikueue cluster is only managed by one admission check controller.


Why this limitation is required?

ravisantoshgudimetla · 2025-08-29T19:59:11Z

Is this something actively being worked on? If not, I can take this over as we have some use-cases at Apple that can benefit from this approach.

khrm · 2025-09-01T09:46:53Z

@ravisantoshgudimetla I am working on this. Either this or #5981 should be done soon.

k8s-triage-robot · 2025-10-01T10:16:58Z

The Kubernetes project currently lacks enough active contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

Mark this PR as fresh with /remove-lifecycle rotten
Close this PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-ci-robot · 2025-10-01T10:17:07Z

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

mimowo · 2025-10-01T10:51:58Z

/close

As per: #2405 (comment)

k8s-ci-robot · 2025-10-01T10:52:04Z

@mimowo: Closed this PR.

In response to this:

/close

As per: #2405 (comment)

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. kind/documentation Categorizes issue or PR as related to documentation. labels Jun 20, 2024

k8s-ci-robot requested review from kerthcet and trasc June 20, 2024 13:58

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jun 20, 2024

k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jun 20, 2024

mimowo mentioned this pull request Jun 20, 2024

Provide a way to implement a MultiKueue support for the custom Jobs #2349

Closed

5 tasks

mszadkow force-pushed the doc/add-custom-job-support branch from a6227d9 to 10fc776 Compare June 21, 2024 13:43

trasc reviewed Jun 25, 2024

View reviewed changes

mszadkow marked this pull request as ready for review June 25, 2024 10:10

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 25, 2024

k8s-ci-robot requested review from denkensk and mimowo June 25, 2024 10:10

k8s-ci-robot requested review from alculquicondor and tenzen-y June 25, 2024 10:31

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jun 26, 2024

alculquicondor reviewed Jun 26, 2024

View reviewed changes

dgrove-oss reviewed Jun 26, 2024

View reviewed changes

keps/2349-multikueue-external-custom-job-support/README.md Outdated Show resolved Hide resolved

Update toc

80ef47d

mszadkow force-pushed the doc/add-custom-job-support branch from ec73442 to 80ef47d Compare September 25, 2024 11:24

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Dec 24, 2024

k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Dec 24, 2024

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Mar 24, 2025

k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Apr 23, 2025

k8s-ci-robot removed the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label May 23, 2025

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Aug 21, 2025

tenzen-y mentioned this pull request Aug 22, 2025

[KEP-2349]: Add MultiKueue support for external custom job #5981

Merged

gbenhaim reviewed Aug 22, 2025

View reviewed changes

gbenhaim mentioned this pull request Aug 22, 2025

Make MultiKueue Admission Check Controller Reusable by External Frameworks #6117

Open

k8s-ci-robot closed this Oct 1, 2025

mszadkow deleted the doc/add-custom-job-support branch October 24, 2025 11:15

KEP: MultiKueue external custom Job support #2458

KEP: MultiKueue external custom Job support #2458

Uh oh!

Conversation

mszadkow commented Jun 20, 2024

What type of PR is this?

What this PR does / why we need it:

Which issue(s) this PR fixes:

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Uh oh!

k8s-ci-robot commented Jun 20, 2024

Uh oh!

k8s-ci-robot commented Jun 20, 2024

Uh oh!

netlify bot commented Jun 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for kubernetes-sigs-kueue canceled.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

trasc commented Jun 25, 2024

Uh oh!

alculquicondor commented Jun 26, 2024

Uh oh!

alculquicondor commented Jun 26, 2024

Uh oh!

Uh oh!

Uh oh!

alculquicondor Jun 26, 2024

Choose a reason for hiding this comment

Uh oh!

dgrove-oss left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

alculquicondor commented Jun 26, 2024

Uh oh!

tenzen-y commented Jun 28, 2024

Uh oh!

alculquicondor commented Jun 28, 2024

Uh oh!

mszadkow commented Sep 25, 2024

Uh oh!

k8s-triage-robot commented Dec 24, 2024

Uh oh!

dgrove-oss commented Dec 24, 2024

Uh oh!

k8s-triage-robot commented Mar 24, 2025

Uh oh!

k8s-triage-robot commented Apr 23, 2025

Uh oh!

yibozhuang commented May 23, 2025

Uh oh!

k8s-triage-robot commented Aug 21, 2025

Uh oh!

gbenhaim Aug 22, 2025

Choose a reason for hiding this comment

Uh oh!

ravisantoshgudimetla commented Aug 29, 2025

Uh oh!

khrm commented Sep 1, 2025

Uh oh!

k8s-triage-robot commented Oct 1, 2025

Uh oh!

k8s-ci-robot commented Oct 1, 2025

Uh oh!

mimowo commented Oct 1, 2025

Uh oh!

k8s-ci-robot commented Oct 1, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

netlify bot commented Jun 20, 2024 •

edited

Loading