add argo integration based on pod integration #3897
Conversation
k8s-ci-robot
added
kind/documentation
k8s-ci-robot
added
the
size/L
✅ Deploy Preview for kubernetes-sigs-kueue ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
|
/uncc |
| integrations: | ||
| frameworks: | ||
| - "pod" | ||
| podOptions: |
There was a problem hiding this comment.
Instead of podOptions we are migrating to use managedJobsNamespaceSelector as a more generic solution, also for plain pods. Could you update as currently in https://kueue.sigs.k8s.io/docs/tasks/run/plain_pods/
cc @dgrove-oss
There was a problem hiding this comment.
I'll just link to that section.
| it should be specified in the `spec.podMetadata` section of the Workflow configuration. | ||
|
|
||
| ```yaml | ||
| apiVersion: argoproj.io/v1alpha1 |
There was a problem hiding this comment.
please more the examples to https://github.com/kubernetes-sigs/kueue/tree/main/site/static/examples/jobs for consistency as a single source of examples easy to run.
| container: | ||
| image: docker/whalesay | ||
| command: [cowsay] | ||
| args: ["{{inputs.parameters.message}}"] |
There was a problem hiding this comment.
does it need to be expanded to make the example runnable?
There was a problem hiding this comment.
I think this is runnable.
|
|
||
| This guide is for [batch users](/docs/tasks#batch-user) that have a basic understanding of Kueue. For more information, see [Kueue's overview](/docs/overview). | ||
|
|
||
| As of the writing of this doc, Kueue doesn't support Argo Workflows [Workflow](https://argo-workflows.readthedocs.io/en/latest/workflow-concepts/) resources directly, |
There was a problem hiding this comment.
| As of the writing of this doc, Kueue doesn't support Argo Workflows [Workflow](https://argo-workflows.readthedocs.io/en/latest/workflow-concepts/) resources directly, | |
| Currently Kueue doesn't support Argo Workflows [Workflow](https://argo-workflows.readthedocs.io/en/latest/workflow-concepts/) resources directly, |
nit
| As of the writing of this doc, Kueue doesn't support Argo Workflows [Workflow](https://argo-workflows.readthedocs.io/en/latest/workflow-concepts/) resources directly, | ||
| but you can take advantage of the ability for Kueue to [manage plain pods](/docs/tasks/run_plain_pods) to integrate them. | ||
|
|
||
| 1. By default, the integration for `v1/pod` is not enabled. |
There was a problem hiding this comment.
Please wrap the step in "Before you begin" for consistency, example: https://kueue.sigs.k8s.io/docs/tasks/run/jobsets/. And please add a point about installing the Argo Workflow, along with a link indicating how to do it.
site/content/en/docs/tasks/_index.md
Outdated
| @@ -44,6 +44,7 @@ As a batch user, you can learn how to: | |||
| - [Run a Kueue managed plain Pod](run/plain_pods). | |||
| - [Run a Kueue managed JobSet](run/jobsets). | |||
| - [Submit jobs to MultiKueue](run/multikueue). | |||
| - [Run an argo workflow](run/argo_workload) | |||
There was a problem hiding this comment.
This is great. We also have a similar way of support for the Notebook integration: #3878. I think we could have a dedicated section for them like "Run workloads with Pods managed by Kueue", with this doc being the first of this kind, but Notebook a potential follow up. WDYT @kannon92 @varshaprasad96 @tenzen-y ?
There was a problem hiding this comment.
I also have #3898. So I think this is a good idea to document examples that use pod integration.
There was a problem hiding this comment.
great, I missed that, I would put them together in one sub-folder. We can maybe first merge this one, then align the other.
kannon92
force-pushed
the
argo-integration
branch
from
8871d9d to
c9a302f
Compare
|
@mimowo PTAL |
|
/approve |
k8s-ci-robot
added
the
do-not-merge/hold
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: kannon92, mimowo The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
k8s-ci-robot
added
the
approved
| @@ -0,0 +1,10 @@ | |||
| --- | |||
| title: "Pod Based Workloads" | |||
There was a problem hiding this comment.
Maybe we could change the title to: "External integrations", or something similar? Pod Based Workloads can be a little misleading, since we already have Plain Pods section
There was a problem hiding this comment.
I agree it is worth thinking about a good name, but we already have a concept of "external integrations" / custom jobs: https://kueue.sigs.k8s.io/docs/tasks/dev/integrate_a_custom_job/
So, it could be misleading too.
There was a problem hiding this comment.
OTOH, now thinking more about this, Plain Pods is one of the mechanisms for supporting external frameworks, as suggested here: #3897 (review).
So, I'm ok with the more generic section like "External integrations", then it should also probably group https://kueue.sigs.k8s.io/docs/tasks/run/flux_miniclusters/ and https://kueue.sigs.k8s.io/docs/tasks/run/wrapped_custom_workload/, and refactor the structure. WDYT @PBundyra @kannon92 ?
|
Other than the one comment I left, LGTM! Feel free to unhold your PR after addressing it. |
|
/lgtm |
k8s-ci-robot
added
the
lgtm
|
LGTM label has been added. Git tree hash: c034e48c52ccc8eb568493ca2b67a9b23abdb2fe
|
There was a problem hiding this comment.
LGTM overall, please additionally extend the list of options in https://kueue.sigs.k8s.io/docs/tasks/dev/integrate_a_custom_job/ by adding a point like "Leverage the Plain Pods integration", linking to the new tasks page.
kannon92
force-pushed
the
argo-integration
branch
from
c9a302f to
cfce9d7
Compare
k8s-ci-robot
removed
the
lgtm
|
New changes are detected. LGTM label has been removed. |
|
/hold Addressing #3897 (comment). Need to confirm that this worked. |
kannon92
force-pushed
the
argo-integration
branch
from
19d0da9 to
44a7ee9
Compare
kannon92
force-pushed
the
argo-integration
branch
from
7eb4431 to
88d0551
Compare
There was a problem hiding this comment.
Actually, I would like to consider the following sturcture:
Run Workloads:
- Kubernetes Jobs
- Kubernetes CronJobs
- AppWrappers
- RayClusters
- RayJobs
- Deployment
- StatefulSet
- Plain Pods
- Kubeflow Jobs
- Python
- Jobsets
- External Frameworks
* Using AppWrapper (renamed from Custom Workload)
* Flux MiniClusters
* Argo Workflows
- Multi-Cluster
I don't think we need more structure under "External frameworks" at this point, because there are only three items underneath, and extra layers mean more clicking to expand.
site/content/en/docs/tasks/_index.md
Outdated
| @@ -63,6 +63,7 @@ As a platform developer, you can learn how to: | |||
| - [Integrate a custom Job with Kueue](dev/integrate_a_custom_job). | |||
| - [Enable pprof endpoints](dev/enabling_pprof_endpoints). | |||
| - [Develop a custom AdmissionCheck Controller](dev/develop-acc). | |||
| - [Integration a custom workload with existing frameworks](dev/external_frameworks) | |||
There was a problem hiding this comment.
| - [Integration a custom workload with existing frameworks](dev/external_frameworks) | |
| - [Integrate a custom workload with Kueue using built-in frameworks](dev/external_frameworks) |
Also, move it up just below "Integrate a custom Job with Kueue.
site/content/en/docs/tasks/_index.md
Outdated
| - [Submit Kueue jobs from Python](run/python_jobs). | ||
| - [Run a Kueue managed plain Pod](run/plain_pods). | ||
| - [Run a Kueue managed JobSet](run/jobsets). | ||
| - [Submit jobs to MultiKueue](run/multikueue). | ||
| - [Support external workloads](run/external_workloads). | ||
| Kueue allows one to use existing integrations to support external workloads. |
There was a problem hiding this comment.
| Kueue allows one to use existing integrations to support external workloads. | |
| Kueue allows one to use built-in integrations (such as Pods or Jobs) to run external workloads. |
| ### Job-based Workloads Integration | ||
| - [Run a Flux Miniclusters using job integration](/docs/tasks/run/external_workloads/flux_miniclusters). | ||
|
|
||
| ### Pod-Based Workloads Integration | ||
| - [Run an Argo Workflow using pod integration](/docs/tasks/run/external_workloads/pod_based_workloads/argo_workflow). |
There was a problem hiding this comment.
I would suggest to combine the two, because conceptually this is the same mechanism:
| ### Job-based Workloads Integration | |
| - [Run a Flux Miniclusters using job integration](/docs/tasks/run/external_workloads/flux_miniclusters). | |
| ### Pod-Based Workloads Integration | |
| - [Run an Argo Workflow using pod integration](/docs/tasks/run/external_workloads/pod_based_workloads/argo_workflow). | |
| ### Integrations based on built-in frameworks | |
| - [Run a Flux Miniclusters using job integration](/docs/tasks/run/external_workloads/flux_miniclusters). | |
| - [Run an Argo Workflow using pod integration](/docs/tasks/run/external_workloads/pod_based_workloads/argo_workflow). |
| @@ -1,6 +1,5 @@ | |||
| --- | |||
| title: "Run An AppWrapper" | |||
| linkTitle: "AppWrappers" | |||
There was a problem hiding this comment.
No need to remove. Let's also keep AppWrapper where it was as it is a built-in framework currently. We have another page for showing how to use AppWrapper for custom Jobs.
There was a problem hiding this comment.
No need to remove. Let's also keep AppWrapper where it was as it is a built-in framework currently. We have another page for showing how to use AppWrapper for custom Jobs.
I think this makes sense.
Once Kueue 0.11 comes out, we plan to simplify the AppWrapper controller, remove the code that is no longer needed since it is now a built-in framework, and make a new release of AppWrapper. As part of doing that, I will revise the Kueue documention and make sure it refers to the right historic pieces of external-framework code in the AppWrapper 1.0 git.
kannon92
force-pushed
the
argo-integration
branch
from
fd0e227 to
9e4fe58
Compare
kannon92
force-pushed
the
argo-integration
branch
from
9e4fe58 to
30558fc
Compare
What type of PR is this?
/kind documentation
What this PR does / why we need it:
Add an integration with Kueue and Argo Workflows
Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
Credit goes to sam-leitch-oxb.
Nice work!
Does this PR introduce a user-facing change?