generated from kubernetes/kubernetes-template-project
-
Notifications
You must be signed in to change notification settings - Fork 286
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
6 changed files
with
134 additions
and
168 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
10 changes: 10 additions & 0 deletions
10
site/content/en/docs/tasks/run/pod-based-workloads/_index.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
--- | ||
title: "Pod Based Workloads" | ||
linkTitle: "Pod-based-workloads" | ||
weight: 8 | ||
date: 2025-01-17 | ||
description: > | ||
Using Pod Integration to support external integrations | ||
--- | ||
|
||
This page explains how to use pod integration to support external workloads. |
51 changes: 51 additions & 0 deletions
51
site/content/en/docs/tasks/run/pod-based-workloads/argo_workload.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,51 @@ | ||
--- | ||
title: "Run An Argo Workflow" | ||
date: 2024-01-03 | ||
weight: 7 | ||
description: > | ||
Integrate Kueue with Argo Workflows. | ||
--- | ||
|
||
This page shows how to leverage Kueue's scheduling and resource management capabilities when running [Argo Workflows](https://argo-workflows.readthedocs.io/en/latest/). | ||
|
||
This guide is for [batch users](/docs/tasks#batch-user) that have a basic understanding of Kueue. For more information, see [Kueue's overview](/docs/overview). | ||
|
||
Currently Kueue doesn't support Argo Workflows [Workflow](https://argo-workflows.readthedocs.io/en/latest/workflow-concepts/) resources directly, | ||
but you can take advantage of the ability for Kueue to [manage plain pods](/docs/tasks/run_plain_pods) to integrate them. | ||
|
||
## Before you begin | ||
|
||
1. Learn how to [install Kueue with a custom manager configuration](/docs/installation/#install-a-custom-configured-released-version). | ||
|
||
2. Follow steps in [Run Plain Pods](/docs/tasks/run/plain_pods/#before-you-begin) | ||
to learn how to enable and configure the `v1/pod` integration. | ||
|
||
3. Install [Argo Workflows](https://argo-workflows.readthedocs.io/en/latest/installation/#installation) | ||
|
||
## Workflow definition | ||
|
||
### a. Targeting a single LocalQueue | ||
|
||
If you want the entire workflow to target a single [local queue](/docs/concepts/local_queue), | ||
it should be specified in the `spec.podMetadata` section of the Workflow configuration. | ||
|
||
{{< include "examples/pod-based-workloads/workflow-single-queue.yaml" "yaml" >}} | ||
|
||
### b. Targeting a different LocalQueue per template | ||
|
||
If prefer to target a different [local queue](/docs/concepts/local_queue) for each step of your Workflow, | ||
you can define the queue in the `spec.templates[].metadata` section of the Workflow configuration. | ||
|
||
In this example `hello1` and `hello2a` will target `user-queue` and `hello2b` will | ||
target `user-queue-2`. | ||
|
||
{{< include "examples/pod-based-workloads/workflow-queue-per-template.yaml" "yaml" >}} | ||
|
||
### c. Limitations | ||
|
||
- Kueue will only manage pods created by Argo Workflows. It does not manage the Argo Workflows resources in any way. | ||
- Each pod in a Workflow will create a new Workload resource and must wait for admission by Kueue. | ||
- There is no way to ensure that a Workflow will complete before it is started. If one step of a multi-step Workflow does not have | ||
available quota, Argo Workflows will run all previous steps and then wait for quota to become available. | ||
- Kueue does not understand Argo Workflows `suspend` flag and will not manage it. | ||
- Kueue does not manage `suspend`, `http`, or `resource` template types since they do not create pods. |
52 changes: 52 additions & 0 deletions
52
site/static/examples/pod-based-workloads/workflow-queue-per-template.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,52 @@ | ||
apiVersion: argoproj.io/v1alpha1 | ||
kind: Workflow | ||
metadata: | ||
generateName: steps- | ||
spec: | ||
entrypoint: hello-hello-hello | ||
|
||
templates: | ||
- name: hello-hello-hello | ||
steps: | ||
- - name: hello1 # hello1 is run before the following steps | ||
template: whalesay | ||
arguments: | ||
parameters: | ||
- name: message | ||
value: "hello1" | ||
- - name: hello2a # double dash => run after previous step | ||
template: whalesay | ||
arguments: | ||
parameters: | ||
- name: message | ||
value: "hello2a" | ||
- name: hello2b # single dash => run in parallel with previous step | ||
template: whalesay-queue-2 | ||
arguments: | ||
parameters: | ||
- name: message | ||
value: "hello2b" | ||
|
||
- name: whalesay | ||
metadata: | ||
labels: | ||
kueue.x-k8s.io/queue-name: user-queue # Pods from this template will target user-queue | ||
inputs: | ||
parameters: | ||
- name: message | ||
container: | ||
image: docker/whalesay | ||
command: [cowsay] | ||
args: ["{{inputs.parameters.message}}"] | ||
|
||
- name: whalesay-queue-2 | ||
metadata: | ||
labels: | ||
kueue.x-k8s.io/queue-name: user-queue-2 # Pods from this template will target user-queue-2 | ||
inputs: | ||
parameters: | ||
- name: message | ||
container: | ||
image: docker/whalesay | ||
command: [cowsay] | ||
args: ["{{inputs.parameters.message}}"] |
19 changes: 19 additions & 0 deletions
19
site/static/examples/pod-based-workloads/workflow-single-queue.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
apiVersion: argoproj.io/v1alpha1 | ||
kind: Workflow | ||
metadata: | ||
generateName: hello-world- | ||
spec: | ||
entrypoint: whalesay | ||
podMetadata: | ||
labels: | ||
kueue.x-k8s.io/queue-name: user-queue # All pods will target user-queue | ||
templates: | ||
- name: whalesay | ||
container: | ||
image: docker/whalesay | ||
command: [ cowsay ] | ||
args: [ "hello world" ] | ||
resources: | ||
limits: | ||
memory: 32Mi | ||
cpu: 100m |