Skip to content

Commit f01d373

Browse files
feat: add rancher-terraform and import-existing-vm (#610)
* feat: add rancher-terraform and import-existing-vm Signed-off-by: PoAn Yang <[email protected]> --------- Signed-off-by: PoAn Yang <[email protected]> Co-authored-by: Jillian <[email protected]>
1 parent 196d9bd commit f01d373

File tree

11 files changed

+689
-7
lines changed

11 files changed

+689
-7
lines changed

docs/rancher/csi-driver.md

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -276,3 +276,29 @@ Now you can create a new StorageClass that you intend to use in your guest Kuber
276276
:::
277277

278278
1. You can now create a PVC based on this new **StorageClass**, which utilizes the **Host StorageClass** to provision volumes on the bare-metal Harvester cluster.
279+
280+
## Upgrade the CSI Driver
281+
282+
### Upgrade RKE2
283+
284+
To upgrade the CSI driver, use the Rancher UI to upgrade RKE2. Ensure the new RKE2 version supports/bundled with the updated CSI driver version.
285+
286+
1. Go to **** > **Cluster Management**.
287+
288+
1. Find the guest cluster that you want to upgrade and select **⋮** > **Edit Config**.
289+
290+
1. Select **Kubernetes Version**.
291+
292+
1. Click **Save**.
293+
294+
### Upgrade RKE and K3s
295+
296+
You can upgrade RKE and K3s using the Rancher UI.
297+
298+
1. Go to **** > **RKE/K3s Cluster** > **Apps** > **Installed Apps**.
299+
300+
1. Find the CSI driver chart and select **⋮** > **Edit/Upgrade**.
301+
302+
1. Select **Version**.
303+
304+
1. Select **Next** > **Update**.

docs/rancher/import-existing-vm.md

Lines changed: 143 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,143 @@
1+
---
2+
sidebar_position: 8
3+
sidebar_label: Import Existing Cluster built on Harvester VM
4+
title: "Import Existing Cluster built on Harvester VM"
5+
keywords:
6+
- Harvester
7+
- harvester
8+
- Rancher
9+
- rancher
10+
---
11+
12+
Rancher allows you to import existing Harvester VMs in which you installed Kubernetes.
13+
14+
## Deployment
15+
16+
### Prerequisites
17+
18+
- The Kubernetes cluster is built on top of Harvester VMs.
19+
20+
### Deploy Guest Clusters on Harvester VMs
21+
22+
1. Generate the csi-driver cloud-config file using the [generate_addon_csi.sh](https://raw.githubusercontent.com/harvester/harvester-csi-driver/master/deploy/generate_addon_csi.sh) script, which is available in the [harvester/harvester-csi-driver](https://github.com/harvester/harvester-csi-driver) repository.
23+
24+
Example:
25+
26+
```shell
27+
./generate_addon_csi.sh <serviceaccount name> <namespace> RKE2
28+
```
29+
30+
The generated output will be similar to the following one:
31+
```yaml
32+
########## cloud-config ############
33+
apiVersion: v1
34+
clusters:
35+
- cluster: <token>
36+
server: https://<YOUR HOST HARVESTER VIP>:6443
37+
name: default
38+
contexts:
39+
- context:
40+
cluster: default
41+
namespace: default
42+
user: rke2-guest-01-default-default
43+
name: rke2-guest-01-default-default
44+
current-context: rke2-guest-01-default-default
45+
kind: Config
46+
preferences: {}
47+
users:
48+
- name: rke2-guest-01-default-default
49+
user:
50+
token: <token>
51+
52+
########## cloud-init user data ############
53+
write_files:
54+
- encoding: b64
55+
content: YXBpVmVyc2lvbjogdjEKY2x1c3RlcnM6Ci0gY2x1c3RlcjoKICAgIGNlcnRpZmljYXRlLWF1dGhvcml0eS1kYXRhOiBMUzB0TFMxQ1JVZEpUaUJEUlZKVVNVWkpRMEZVUlMwdExTMHRDazFKU1VKbFZFTkRRVklyWjBGM1NVSkJaMGxDUVVSQlMwSm5aM0ZvYTJwUFVGRlJSRUZxUVd0TlUwbDNTVUZaUkZaUlVVUkVRbXg1WVRKVmVVeFlUbXdLWTI1YWJHTnBNV3BaVlVGNFRtcG5NVTE2VlhoT1JGRjNUVUkwV0VSVVNYcE5SRlY1VDFSQk5VMVVRVEJOUm05WVJGUk5lazFFVlhsT2FrRTFUVlJCTUFwTlJtOTNTa1JGYVUxRFFVZEJNVlZGUVhkM1dtTnRkR3hOYVRGNldsaEtNbHBZU1hSWk1rWkJUVlJaTkU1VVRURk5WRkV3VFVSQ1drMUNUVWRDZVhGSENsTk5ORGxCWjBWSFEwTnhSMU5OTkRsQmQwVklRVEJKUVVKSmQzRmFZMDVTVjBWU2FsQlVkalJsTUhFMk0ySmxTSEZEZDFWelducGtRa3BsU0VWbFpHTUtOVEJaUTNKTFNISklhbWdyTDJab2VXUklNME5ZVURNeFZXMWxTM1ZaVDBsVGRIVnZVbGx4YVdJMGFFZE5aekpxVVdwQ1FVMUJORWRCTVZWa1JIZEZRZ292ZDFGRlFYZEpRM0JFUVZCQ1owNVdTRkpOUWtGbU9FVkNWRUZFUVZGSUwwMUNNRWRCTVZWa1JHZFJWMEpDVWpaRGEzbEJOSEZqYldKSlVESlFWVW81Q2xacWJWVTNVV2R2WjJwQlMwSm5aM0ZvYTJwUFVGRlJSRUZuVGtsQlJFSkdRV2xCZUZKNU4xUTNRMVpEYVZWTVdFMDRZazVaVWtWek1HSnBZbWxVSzJzS1kwRnhlVmt5Tm5CaGMwcHpMM2RKYUVGTVNsQnFVVzVxZEcwMVptNTZWR3AxUVVsblRuTkdibFozWkZRMldXWXpieTg0ZFRsS05tMWhSR2RXQ2kwdExTMHRSVTVFSUVORlVsUkpSa2xEUVZSRkxTMHRMUzBLCiAgICBzZXJ2ZXI6IGh0dHBzOi8vMTkyLjE2OC4wLjEzMTo2NDQzCiAgbmFtZTogZGVmYXVsdApjb250ZXh0czoKLSBjb250ZXh0OgogICAgY2x1c3RlcjogZGVmYXVsdAogICAgbmFtZXNwYWNlOiBkZWZhdWx0CiAgICB1c2VyOiBya2UyLWd1ZXN0LTAxLWRlZmF1bHQtZGVmYXVsdAogIG5hbWU6IHJrZTItZ3Vlc3QtMDEtZGVmYXVsdC1kZWZhdWx0CmN1cnJlbnQtY29udGV4dDogcmtlMi1ndWVzdC0wMS1kZWZhdWx0LWRlZmF1bHQKa2luZDogQ29uZmlnCnByZWZlcmVuY2VzOiB7fQp1c2VyczoKLSBuYW1lOiBya2UyLWd1ZXN0LTAxLWRlZmF1bHQtZGVmYXVsdAogIHVzZXI6CiAgICB0b2tlbjogZXlKaGJHY2lPaUpTVXpJMU5pSXNJbXRwWkNJNklreGhUazQxUTBsMWFsTnRORE5TVFZKS00waE9UbGszTkV0amNVeEtjM1JSV1RoYVpUbGZVazA0YW1zaWZRLmV5SnBjM01pT2lKcmRXSmxjbTVsZEdWekwzTmxjblpwWTJWaFkyTnZkVzUwSWl3aWEzVmlaWEp1WlhSbGN5NXBieTl6WlhKMmFXTmxZV05qYjNWdWRDOXVZVzFsYzNCaFkyVWlPaUprWldaaGRXeDBJaXdpYTNWaVpYSnVaWFJsY3k1cGJ5OXpaWEoyYVdObFlXTmpiM1Z1ZEM5elpXTnlaWFF1Ym1GdFpTSTZJbkpyWlRJdFozVmxjM1F0TURFdGRHOXJaVzRpTENKcmRXSmxjbTVsZEdWekxtbHZMM05sY25acFkyVmhZMk52ZFc1MEwzTmxjblpwWTJVdFlXTmpiM1Z1ZEM1dVlXMWxJam9pY210bE1pMW5kV1Z6ZEMwd01TSXNJbXQxWW1WeWJtVjBaWE11YVc4dmMyVnlkbWxqWldGalkyOTFiblF2YzJWeWRtbGpaUzFoWTJOdmRXNTBMblZwWkNJNkltTXlZak5sTldGaExUWTBNMlF0TkRkbU1pMDROemt3TFRjeU5qWXpNbVl4Wm1aaU5pSXNJbk4xWWlJNkluTjVjM1JsYlRwelpYSjJhV05sWVdOamIzVnVkRHBrWldaaGRXeDBPbkpyWlRJdFozVmxjM1F0TURFaWZRLmFRZmU1d19ERFRsSWJMYnUzWUVFY3hmR29INGY1VnhVdmpaajJDaWlhcXB6VWI0dUYwLUR0cnRsa3JUM19ZemdXbENRVVVUNzNja1BuQmdTZ2FWNDhhdmlfSjJvdUFVZC04djN5d3M0eXpjLVFsTVV0MV9ScGJkUURzXzd6SDVYeUVIREJ1dVNkaTVrRWMweHk0X0tDQ2IwRHQ0OGFoSVhnNlMwRDdJUzFfVkR3MmdEa24wcDVXUnFFd0xmSjdEbHJDOFEzRkNUdGhpUkVHZkUzcmJGYUdOMjdfamR2cUo4WXlJQVd4RHAtVHVNT1pKZUNObXRtUzVvQXpIN3hOZlhRTlZ2ZU05X29tX3FaVnhuTzFEanllbWdvNG9OSEpzekp1VWliRGxxTVZiMS1oQUxYSjZXR1Z2RURxSTlna1JlSWtkX3JqS2tyY3lYaGhaN3lTZ3o3QQo=
56+
owner: root:root
57+
path: /var/lib/rancher/rke2/etc/config-files/cloud-provider-config
58+
permissions: '0644'
59+
```
60+
61+
1. Generate the cloud-provider cloud-config using the [generate_addon.sh](https://raw.githubusercontent.com/harvester/cloud-provider-harvester/master/deploy/generate_addon.sh) script, which is available in the [harvester/cloud-provider-harvester](https://github.com/harvester/cloud-provider-harvester) repository.
62+
63+
Example:
64+
65+
```shell
66+
./generate_addon.sh <serviceaccount name> <namespace>
67+
```
68+
69+
The generated output will be similar to the following one:
70+
The output will look as follows:
71+
72+
```yaml
73+
########## cloud config ############
74+
apiVersion: v1
75+
clusters:
76+
- cluster:
77+
certificate-authority-data: <CACERT>
78+
server: https://HARVESTER-ENDPOINT/k8s/clusters/local
79+
name: local
80+
contexts:
81+
- context:
82+
cluster: local
83+
namespace: default
84+
user: harvester-cloud-provider-default-local
85+
name: harvester-cloud-provider-default-local
86+
current-context: harvester-cloud-provider-default-local
87+
kind: Config
88+
preferences: {}
89+
users:
90+
- name: harvester-cloud-provider-default-local
91+
user:
92+
token: <TOKEN>
93+
94+
########## cloud-init user data ############
95+
write_files:
96+
- encoding: b64
97+
content: <CONTENT>
98+
owner: root:root
99+
path: /etc/kubernetes/cloud-config
100+
permissions: '0644'
101+
```
102+
103+
1. Create VM with two cloud-config files.
104+
105+
![VM with cloud-config](/img/v1.4/rancher/vm-with-cloud-config.png)
106+
107+
1. Install RKE2 in the VM.
108+
109+
```shell
110+
sudo mkdir -p /etc/rancher/rke2
111+
echo "cni: calico
112+
disable-kube-proxy: false
113+
etcd-expose-metrics: false" | sudo tee /etc/rancher/rke2/config.yaml
114+
curl -sfL https://get.rke2.io | sudo sh -
115+
sudo systemctl enable rke2-server.service
116+
sudo systemctl start rke2-server.service
117+
```
118+
119+
1. Verify that RKE2 is running in the VM.
120+
121+
```shell
122+
sudo /var/lib/rancher/rke2/bin/kubectl --kubeconfig /etc/rancher/rke2/rke2.yaml get nodes
123+
```
124+
125+
1. Import the VM into Rancher.
126+
127+
On the Rancher UI, go to **Cluster Management** > **Clusters** > **Import Existing** > **Generic** > **Create**.
128+
129+
![Import Exisging Cluster](/img/v1.4/rancher/import-existing-cluster.png)
130+
131+
```shell
132+
# Run the command in the VM
133+
curl --insecure -sfL https://192.168.0.181:6443/v3/import/g5p2g2gtxw4564nktdl4nr5cwwvtwqp9zxd6dmhm5nc7vpnxmr9cfk_c-m-mzf28skd.yaml | sudo /var/lib/rancher/rke2/bin/kubectl --kubeconfig /etc/rancher/rke2/rke2.yaml apply -f -
134+
```
135+
136+
1. Install the Harvester Cloud Provider.
137+
138+
On the RKE2 Cluster Dashboard, go to **Apps** > **Charts** > **Harvester Cloud Provider** > **Install**.
139+
140+
1. Install the Harvester CSI Driver.
141+
142+
On the RKE2 Cluster Dashboard, go to **Apps** > **Charts** > **Harvester CSI Driver** > **Install**.
143+

docs/rancher/rancher-terraform.md

Lines changed: 184 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,184 @@
1+
---
2+
sidebar_position: 7
3+
sidebar_label: Rancher Terraform
4+
title: "Rancher Terraform"
5+
keywords:
6+
- Harvester
7+
- harvester
8+
- Rancher
9+
- rancher
10+
- Terraform
11+
- terraform
12+
description: Rancher Terraform allows administrators to create and manage RKE2 guest clusters using Terraform.
13+
---
14+
15+
The [Rancher Terraform Provider](https://registry.terraform.io/providers/rancher/rancher2/) allows administrators to create and manage RKE2 guest clusters using Terraform.
16+
17+
## Deployment
18+
19+
### Prerequisites
20+
21+
- The Kubernetes cluster is built on top of Harvester VMs.
22+
- The Harvester VMs that run as guest Kubernetes nodes are in the same namespace.
23+
24+
25+
### Deploy Guest Clusters Using the Rancher Terraform Provider
26+
27+
1. Create an API key.
28+
29+
On the Rancher UI, go to **Account & API Keys** > **Create API key** > **Create**.
30+
31+
![Create API Key](/img/v1.4/rancher/create-api-key.png)
32+
![Access & Secret Keys](/img/v1.4/rancher/access-and-secret-keys.png)
33+
34+
1. Obtain the Harvester cluster ID.
35+
36+
On the Rancher UI, go to **Virtualization Management** > **Manage** > **Related Resources** > **Mgmt Cluster Name**.
37+
38+
![Harvester Cluster ID](/img/v1.4/rancher/harvester-cluster-id.png)
39+
40+
1. Obtain the kubeconfig for the Harvester Cloud Provider and the Harvester CSI Driver.
41+
42+
43+
```shell
44+
# Generate harvester cloud provider kubeconfig
45+
RANCHER_SERVER_URL="<RANCHER_SERVER_URL>" # Pure server URL like https://192.168.0.181:6443
46+
RANCHER_ACCESS_KEY="<RANCHER_ACCESS_KEY>"
47+
RANCHER_SECRET_KEY="<RANCHER_SECRET_KEY>"
48+
HARVESTER_CLUSTER_ID="<HARVESTER_CLUSTER_ID>"
49+
CLUSTER_NAME="rke2-demo"
50+
curl -k -X POST ${RANCHER_SERVER_URL}/k8s/clusters/${HARVESTER_CLUSTER_ID}/v1/harvester/kubeconfig \
51+
-H 'Content-Type: application/json' \
52+
-u ${RANCHER_ACCESS_KEY}:${RANCHER_SECRET_KEY} \
53+
-d '{"clusterRoleName": "harvesterhci.io:cloudprovider", "namespace": "default", "serviceAccountName": "'${CLUSTER_NAME}'"}' | xargs | sed 's/\\n/\n/g' > ${CLUSTER_NAME}-kubeconfig
54+
```
55+
56+
1. Prepare a `provider.tf` file with the following content:
57+
58+
```hcl
59+
terraform {
60+
required_providers {
61+
rancher2 = {
62+
source = "rancher/rancher2"
63+
version = "4.2.0"
64+
}
65+
}
66+
}
67+
68+
# Configure the Rancher2 provider to admin
69+
provider "rancher2" {
70+
api_url = "<api_url>" # API Endpoint on Account & API Keys page
71+
access_key = "<access_key>"
72+
secret_key = "<secret_key>"
73+
insecure = true # Set to true if the Rancher server uses a self-signed certificate
74+
}
75+
```
76+
77+
1. Prepare a `main.tf` file with the following content:
78+
79+
```hcl
80+
# Get imported harvester cluster info
81+
data "rancher2_cluster_v2" "harv" {
82+
name = "<harvester_cluster_name_in_rancher>"
83+
}
84+
85+
# Create a new Cloud Credential for an imported Harvester cluster
86+
resource "rancher2_cloud_credential" "harv-cred" {
87+
name = "harv-cred"
88+
harvester_credential_config {
89+
cluster_id = data.rancher2_cluster_v2.harv.cluster_v1_id
90+
cluster_type = "imported"
91+
kubeconfig_content = data.rancher2_cluster_v2.harv.kube_config
92+
}
93+
}
94+
95+
# Create a new rancher2 machine config v2 using harvester node_driver
96+
resource "rancher2_machine_config_v2" "rke2-machine" {
97+
generate_name = "rke2-machine"
98+
harvester_config {
99+
vm_namespace = "default"
100+
cpu_count = "2"
101+
memory_size = "4"
102+
disk_info = <<EOF
103+
{
104+
"disks": [{
105+
"imageName": "default/<vmimage-name>",
106+
"size": 15,
107+
"bootOrder": 1
108+
}]
109+
}
110+
EOF
111+
network_info = <<EOF
112+
{
113+
"interfaces": [{
114+
"networkName": "default/<network-name>"
115+
}]
116+
}
117+
EOF
118+
ssh_user = "<ssh_user>"
119+
user_data = <<EOF
120+
package_update: true
121+
packages:
122+
- qemu-guest-agent
123+
- iptables
124+
runcmd:
125+
- - systemctl
126+
- enable
127+
- '--now'
128+
- qemu-guest-agent.service
129+
EOF
130+
}
131+
}
132+
133+
resource "rancher2_cluster_v2" "rke2-demo" {
134+
name = "rke2-demo"
135+
kubernetes_version = "v1.28.10+rke2r1"
136+
rke_config {
137+
machine_pools {
138+
name = "pool1"
139+
cloud_credential_secret_name = rancher2_cloud_credential.harv-cred.id
140+
control_plane_role = true
141+
etcd_role = true
142+
worker_role = true
143+
quantity = 1
144+
machine_config {
145+
kind = rancher2_machine_config_v2.rke2-machine.kind
146+
name = rancher2_machine_config_v2.rke2-machine.name
147+
}
148+
}
149+
150+
machine_selector_config {
151+
config = yamlencode({
152+
cloud-provider-config = file("${path.module}/rke2-demo-kubeconfig")
153+
cloud-provider-name = "harvester"
154+
})
155+
}
156+
157+
machine_global_config = <<EOF
158+
cni: "calico"
159+
disable-kube-proxy: false
160+
etcd-expose-metrics: false
161+
EOF
162+
163+
upgrade_strategy {
164+
control_plane_concurrency = "1"
165+
worker_concurrency = "1"
166+
}
167+
168+
etcd {
169+
snapshot_schedule_cron = "0 */5 * * *"
170+
snapshot_retention = 5
171+
}
172+
173+
chart_values = <<EOF
174+
harvester-cloud-provider:
175+
clusterName: rke2-demo
176+
cloudConfigPath: /var/lib/rancher/rke2/etc/config-files/cloud-provider-config
177+
EOF
178+
}
179+
}
180+
```
181+
182+
1. Run `terraform init`.
183+
184+
1. Run `terraform apply`.
238 KB
Loading
167 KB
Loading
273 KB
Loading
306 KB
Loading
727 KB
Loading

0 commit comments

Comments
 (0)