Repository navigation
Conversation
da71b9d to
5354ef7
Compare
Terraform for a small GKE cluster that runs the text-to-SQL recipe as published, on its own model, google/gemma-4-e2b: one g2-standard-24 node with two NVIDIA L4 GPUs, one for the LoRA trainer and one for the vLLM sampler. It is the GKE Setup Guide as code, kept to the minimum needed to run the recipe (default network, default node service account), so it is not a production setup. It is two roots, applied in order. 1-infra enables the APIs and creates a zonal GKE Standard cluster with the Filestore CSI driver and the GPU pool, with GKE's own driver install and device plugin off so DRA can hand out the GPUs (zone and gpu_spot are variables, since L4s stock out). 2-cluster is separate because its Helm and kubectl providers can only be configured once the cluster exists. It installs the NVIDIA driver, the NVIDIA DRA driver, and the OpenRL LoRA control plane, and outputs the port-forward and the command that runs the recipe against the cluster. The control plane is the recipe's own overlay, examples/text-to-sql (the LoRA bundle k8s/deploy/lora with Gemma 4 E2B as the default model and a part-of: text-to-sql label), so it stays in sync with the checked-out commit and there is no vendored copy to drift. The run_recipe output runs the recipe's preset with no model override. 2-cluster renders the overlay at plan time with the third-party kbst/kustomization provider (render only, it never talks to the cluster), writes the result to charts/openrl/.tmp/openrl-lora.yaml, and installs it through a local Helm chart with no logic: its only template prints that file with .Files.Get. The file lives inside the chart because Helm can only read files there, and is written with local_sensitive_file only to keep the ~600 KB bundle out of the plan. Helm does not notice a changed file in a local chart, so the release carries the bundle's sha256 as an unused value; a change to the overlay or under k8s/ then shows up in the next plan and the apply upgrades the release, which also deletes objects removed from the overlay. main.tf and the README call out the third-party provider. The configs declare only the non-hashicorp provider sources and pin no versions. .gitignore now ignores Terraform state, .terraform/, lock files, tfvars (commit *.tfvars.example only) and plans, the recipe's artifacts/ run outputs, local credentials, and any .tmp/ directory, which holds the rendered bundle. Signed-off-by: Steve Linde <stevenlinde@google.com>
5354ef7 to
54e65c7
Compare
|
|
||
| data "kustomization_build" "openrl" { | ||
| path = "${path.module}/../../../../examples/text-to-sql" | ||
| } |
There was a problem hiding this comment.
The raw kustomize build keeps api-server:latest and scheduler:latest (only make render VERSION=… sets tags), so images don't follow the checked-out commit. Can we set the tags here, or drop the README's "check out a release tag for a fixed deployment" line?
| resource "google_container_cluster" "openrl" { | ||
| name = "open-rl-ttsql" | ||
| location = var.zone | ||
| initial_node_count = 0 # no default node pool; the GPU pool is below |
There was a problem hiding this comment.
The guide has a CPU pool, here the control plane runs on the GPU node. If GKE adds the nvidia.com/gpu taint, the API server, scheduler and Redis have no toleration and stay Pending. Did a clean apply come up? If the node is tainted, a small CPU pool would fix it.
|
|
||
| ## Tear down | ||
|
|
||
| Destroy stage 2 first: removing the OpenRL bundle deletes its PVC, which deletes the Filestore instance before the cluster goes. |
There was a problem hiding this comment.
Two risks here: 2. worker CRs keep their openrl.io/placement finalizer once the scheduler is gone, so the destroy may hang 2. the Filestore delete is async, so it can outlive the cluster.
|
Thanks for this. The Terraform layout is clean, and the two-root split is correct. My main concern is maintenance. Details follow, most important first. 1. MaintenanceThree paths now render the bundle. The release bundle and the guide use The root copies facts that change often. These are the DRA chart version and values, the driver installer URL, the node labels, the service name, port 9003, the PVC name and the preset name. The guide changed for these facts in #418, #253 and Nothing tests the root. CI runs only ruff. Provider versions are not pinned, and the lock files are gitignored. When a provider or a name changes, the root breaks silently. A user finds the failure during There are four layers to debug. The path is: overlay, Terraform provider, local file, Helm chart, Helm Secret. A change that works with The third-party provider adds risk for an optional feature. The recipe README says that the release bundle runs the recipe without changes. The overlay only sets the default The scope will grow. The path 2. Discovery and coexistence with the guide
3. Correctness (please check)
Suggestions
If you keep the overlay, please do items 2 to 5 at least. Also, render through |
Terraform for a small GKE cluster that runs the text-to-SQL recipe as
published, on its own model, google/gemma-4-e2b: one g2-standard-24 node
with two NVIDIA L4 GPUs, one for the LoRA trainer and one for the vLLM
sampler. It is the GKE Setup Guide as code, kept to the minimum needed to
run the recipe (default network, default node service account), so it is not
a production setup.
It is two roots, applied in order. 1-infra enables the APIs and creates a
zonal GKE Standard cluster with the Filestore CSI driver and the GPU pool,
with GKE's own driver install and device plugin off so DRA can hand out the
GPUs (zone and gpu_spot are variables, since L4s stock out). 2-cluster is
separate because its Helm and kubectl providers can only be configured once
the cluster exists. It installs the NVIDIA driver, the NVIDIA DRA driver,
and the OpenRL LoRA control plane, and outputs the port-forward and the
command that runs the recipe against the cluster.
The control plane is the recipe's own overlay, examples/text-to-sql (the
LoRA bundle k8s/deploy/lora with Gemma 4 E2B as the default model and a
part-of: text-to-sql label), so it stays in sync with the checked-out commit
and there is no vendored copy to drift. The run_recipe output runs the
recipe's preset with no model override.
2-cluster renders the overlay at plan time with the third-party
kbst/kustomization provider (render only, it never talks to the cluster),
writes the result to charts/openrl/.tmp/openrl-lora.yaml, and installs it
through a local Helm chart with no logic: its only template prints that file
with .Files.Get. The file lives inside the chart because Helm can only read
files there, and is written with local_sensitive_file only to keep the ~600
KB bundle out of the plan. Helm does not notice a changed file in a local
chart, so the release carries the bundle's sha256 as an unused value; a
change to the overlay or under k8s/ then shows up in the next plan and the
apply upgrades the release, which also deletes objects removed from the
overlay. main.tf and the README call out the third-party provider.
The configs declare only the non-hashicorp provider sources and pin no
versions. .gitignore now ignores Terraform state, .terraform/, lock files,
tfvars (commit *.tfvars.example only) and plans, the recipe's artifacts/ run
outputs, local credentials, and any .tmp/ directory, which holds the
rendered bundle.