Skip to content

Commit

Permalink
Test
Browse files Browse the repository at this point in the history
  • Loading branch information
trasc committed Oct 18, 2024
1 parent 6149b9b commit 93c9c93
Show file tree
Hide file tree
Showing 3 changed files with 124 additions and 6 deletions.
3 changes: 1 addition & 2 deletions cmd/experimental/demo-acc/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Build the manager binary
FROM golang:1.22 AS builder
FROM golang:1.23 AS builder
ARG TARGETOS
ARG TARGETARCH

Expand All @@ -13,7 +13,6 @@ RUN go mod download

# Copy the go source
COPY cmd/main.go cmd/main.go
COPY api/ api/
COPY internal/controller/ internal/controller/

# Build
Expand Down
121 changes: 117 additions & 4 deletions cmd/experimental/demo-acc/STEP-BY-STEP.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Choose a domain and repo for the project, for the purpose of this demo the proje
## Init the project

```bash
kubebuilder init --domain demo-acc.experimental.kueue.x-k8s.io --repo sigs.k8s.io/kueue/cmd/experimental/demo-acc
kubebuilder init --domain demo-acc.experimental.kueue.x-k8s.io --repo sigs.k8s.io/kueue/cmd/experimental/demo-acc
```

## Scaffold the controllers
Expand Down Expand Up @@ -227,16 +227,129 @@ In `cmd/main.go`
metricsserver "sigs.k8s.io/controller-runtime/pkg/metrics/server"
"sigs.k8s.io/controller-runtime/pkg/webhook"
+ kueueapi "sigs.k8s.io/kueue/apis/kueue/v1beta1"

"sigs.k8s.io/kueue/cmd/experimental/demo-acc/internal/controller"
// +kubebuilder:scaffold:imports
@@ -46,6 +47,7 @@ var (

func init() {
utilruntime.Must(clientgoscheme.AddToScheme(scheme))
+ utilruntime.Must(kueueapi.AddToScheme(scheme))

// +kubebuilder:scaffold:scheme
}
```

## Test the ACC

### Install Kueue

Install kueue as described in its [installation guide](https://kueue.sigs.k8s.io/docs/installation/).

### Build the image

Note: Since at this point the operator is not defying an API, the `COPY api/ api/` line should be removed to be able to build the image.
Note: Since at this point kueue is using go v1.23 update the builder's base image in the Dockerfile.

```bash
make docker-build
```

Make sure the test cluster has access to the built image (it can be pushed in a image repository or directly in a test cluster).


Skip `make install` since we have no API defined.

```bash
make deploy
```

Make sure the controller's manager is properly running

```bash
kubectl get -n demo-acc-system deployments.apps
```

Set up a single cluster queue environment as described [here](https://kueue.sigs.k8s.io/docs/tasks/manage/administer_cluster_quotas/#single-clusterqueue-and-single-resourceflavor-setup).
This can be done with:
```bash
kubectl apply -f https://kueue.sigs.k8s.io/examples/admin/single-clusterqueue-setup.yaml
```

Create an AdmissionCheck managed by demo-acc

```bash
kubectl apply -f - <<EOF
apiVersion: kueue.x-k8s.io/v1beta1
kind: AdmissionCheck
metadata:
name: demo-ac
spec:
controllerName: experimental.kueue.x-k8s.io/demo-acc
EOF
```

It should get marked as `Active`

```bash
kubectl get admissionchecks.kueue.x-k8s.io demo-ac -o=jsonpath='{.status.conditions[?(@.type=="Active")].status}{" -> "}{.status.conditions[?(@.type=="Active")].message}{"\n"}'
```

Should output: `True -> demo-acc is running`

Add the admission check to the `cluster-queue` queue:

```bash
kubectl patch clusterqueues.kueue.x-k8s.io cluster-queue --type='json' -p='[{"op": "add", "path": "/spec/admissionChecks", "value":["demo-ac"]}]'
```

Create a sample-job:

```bash
kubectl create -f https://kueue.sigs.k8s.io/examples/jobs/sample-job.yaml
```

Observe the workload's execution.

Given the job is created at t0
- Since the queue is not used, it will get its quota reserved at t0
- demo-acc will marks its admission check state at t0+1m
- it will be admitted at t0+1m

For example:

```yaml
apiVersion: kueue.x-k8s.io/v1beta1
kind: Workload
metadata:
creationTimestamp: "2024-10-18T12:37:15Z" #t0
#<skip>
spec:
active: true
#<skip>
status:
admission:
clusterQueue: cluster-queue
#<skip>
admissionChecks:
- lastTransitionTime: "2024-10-18T12:38:15Z" #t0+1m
message: The workload is now ready
name: demo-ac
state: Ready
conditions:
- lastTransitionTime: "2024-10-18T12:37:15Z" #t0
message: Quota reserved in ClusterQueue cluster-queue
observedGeneration: 1
reason: QuotaReserved
status: "True"
type: QuotaReserved
- lastTransitionTime: "2024-10-18T12:38:15Z" #t0+1m
message: The workload is admitted
observedGeneration: 1
reason: Admitted
status: "True"
type: Admitted
- #<skip>
```


6 changes: 6 additions & 0 deletions cmd/experimental/demo-acc/config/manager/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -1,2 +1,8 @@
resources:
- manager.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
images:
- name: controller
newName: controller
newTag: latest

0 comments on commit 93c9c93

Please sign in to comment.