-
Notifications
You must be signed in to change notification settings - Fork 14.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Document labels, annotations and taints for JobSet #49217
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -2551,6 +2551,111 @@ Starting in v1.16, this annotation was removed in favor of | |
[Pod Priority](/docs/concepts/scheduling-eviction/pod-priority-preemption/). | ||
{{< /note >}} | ||
|
||
### jobset.sigs.k8s.io/jobset-name | ||
|
||
Type: Label, Annotation | ||
|
||
Example: `jobset.sigs.k8s.io/jobset-name: "my-jobset"` | ||
|
||
Used on: Jobs, Pods | ||
|
||
This label/annotation is used to store the name of the JobSet that a Job or Pod belongs to. | ||
[JobSet](https://jobset.sigs.k8s.io) is an extension API that you can deploy into your Kubernetes cluster. | ||
|
||
### jobset.sigs.k8s.io/replicatedjob-replicas | ||
|
||
Type: Label, Annotation | ||
|
||
Example: `jobset.sigs.k8s.io/replicatedjob-replicas: "5"` | ||
|
||
Used on: Jobs, Pods | ||
|
||
This label/annotation specifies the number of replicas for a ReplicatedJob. | ||
|
||
### jobset.sigs.k8s.io/replicatedjob-name | ||
|
||
Type: Label, Annotation | ||
|
||
Example: `jobset.sigs.k8s.io/replicatedjob-name: "my-replicatedjob"` | ||
|
||
Used on: Jobs, Pods | ||
|
||
This label or annotation stores the name of the replicated job that this Job or Pod is part of. | ||
|
||
### jobset.sigs.k8s.io/job-index | ||
|
||
Type: Label, Annotation | ||
|
||
Example: `jobset.sigs.k8s.io/job-index: "0"` | ||
|
||
Used on: Jobs, Pods | ||
|
||
This label/annotation is set by the JobSet controller on child Jobs and Pods. It contains the index of the Job replica within its parent ReplicatedJob. | ||
|
||
### jobset.sigs.k8s.io/job-key | ||
|
||
Type: Label, Annotation | ||
|
||
Example: `jobset.sigs.k8s.io/job-key: "0f1e93893c4cb372080804ddb9153093cb0d20cefdd37f653e739c232d363feb"` | ||
|
||
Used on: Jobs, Pods | ||
|
||
The JobSet controller sets this label (and also an annotation with the same key) on child Jobs and | ||
Pods of a JobSet. The value is the SHA256 hash of the namespaced Job name. | ||
|
||
### alpha.jobset.sigs.k8s.io/exclusive-topology | ||
|
||
Type: Annotation | ||
|
||
Example: `alpha.jobset.sigs.k8s.io/exclusive-topology: "zone"` | ||
|
||
Used on: JobSets, Jobs | ||
|
||
You can set this label/annotation on a [JobSet](https://jobset.sigs.k8s.io) to ensure exclusive Job | ||
placement per topology group. You can also define this label or annotation on a replicated job | ||
template. Read the documentation for JobSet to learn more. | ||
|
||
### alpha.jobset.sigs.k8s.io/node-selector | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ideally we can mark this deprecated (it should be), but: can wait for a follow up PR. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. we will deprecate when we graduate to stable |
||
|
||
Type: Annotation | ||
|
||
Example: `alpha.jobset.sigs.k8s.io/node-selector: "true"` | ||
|
||
Used on: Jobs, Pods | ||
|
||
This label/annotation can be applied to a JobSet. When it's set, the JobSet controller modifies the Jobs and their corresponding Pods by adding node selectors and tolerations. This ensures exclusive job placement per topology domain, restricting the scheduling of these Pods to specific nodes based on the strategy. | ||
|
||
### alpha.jobset.sigs.k8s.io/namespaced-job | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ideally we can mark this deprecated (it should be), but: can wait for a follow up PR. |
||
|
||
Type: Label | ||
|
||
Example: `alpha.jobset.sigs.k8s.io/namespaced-job: "default_myjobset-replicatedjob-0"` | ||
|
||
Used on: Nodes | ||
|
||
This label is either set manually or automatically (for example, a cluster autoscaler) on the nodes. When `alpha.jobset.sigs.k8s.io/node-selector` is set to `"true"`, the JobSet controller adds a nodeSelector to this node label (along with the toleration to the taint `alpha.jobset.sigs.k8s.io/no-schedule` disucssed next). | ||
|
||
### alpha.jobset.sigs.k8s.io/no-schedule | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ideally we can mark this deprecated (it should be), but: can wait for a follow up PR. |
||
|
||
Type: Taint | ||
|
||
Example: `alpha.jobset.sigs.k8s.io/no-schedule: "NoSchedule"` | ||
|
||
Used on: Nodes | ||
|
||
This taint is either set manually or automatically (for example, a cluster autoscaler) on the nodes. When `alpha.jobset.sigs.k8s.io/node-selector` is set to `"true"`, the JobSet controller adds a toleration to this node taint (along with the node selector to the label `alpha.jobset.sigs.k8s.io/namespaced-job` disucssed previously). | ||
|
||
### jobset.sigs.k8s.io/coordinator | ||
|
||
Type: Annotation, Label | ||
|
||
Example: `jobset.sigs.k8s.io/coordinator: "myjobset-workers-0-0.headless-svc"` | ||
|
||
Used on: Jobs, Pods | ||
|
||
This annotation/label is used on Jobs and Pods to store a stable network endpoint where the coordinator | ||
pod can be reached if the [JobSet](https://jobset.sigs.k8s.io) spec defines the `.spec.coordinator` field. | ||
|
||
## Annotations used for audit | ||
|
||
<!-- sorted by annotation --> | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ideally we can mark this deprecated (it should be), but: can wait for a follow up PR.