Skip to content

feat(k8s): kosmos: add-external-node command #4651

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 17, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
🎲🎲🎲 EXIT CODE: 0 🎲🎲🎲
🟥🟥🟥 STDERR️️ 🟥🟥🟥️
Add an external node to a Kosmos Pool.
This will connect via SSH to the node, download the multicloud configuration script and run it with sudo privileges.
Keep in mind that your external node needs to have wget in order to download the script.

USAGE:
scw k8s pool add-external-node [arg=value ...]

ARGS:
node-ip IP address of the external node
pool-id ID of the Pool the node should be added to
[username=root] Username used for the SSH connection
[region=fr-par] Region to target. If none is passed will use default region from the config

FLAGS:
-h, --help help for add-external-node

GLOBAL FLAGS:
-c, --config string The path to the config file
-D, --debug Enable debug mode
-o, --output string Output format: json or human, see 'scw help output' for more info (default "human")
-p, --profile string The config profile to use
15 changes: 8 additions & 7 deletions cmd/scw/testdata/test-all-usage-k8s-pool-usage.golden
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,16 @@ USAGE:
scw k8s pool <command>

AVAILABLE COMMANDS:
create Create a new Pool in a Cluster
delete Delete a Pool in a Cluster
get Get a Pool in a Cluster
list List Pools in a Cluster
update Update a Pool in a Cluster
upgrade Upgrade a Pool in a Cluster
add-external-node Add an external node to a Kosmos Pool
create Create a new Pool in a Cluster
delete Delete a Pool in a Cluster
get Get a Pool in a Cluster
list List Pools in a Cluster
update Update a Pool in a Cluster
upgrade Upgrade a Pool in a Cluster

WORKFLOW COMMANDS:
wait Wait for a pool to reach a stable state
wait Wait for a pool to reach a stable state

FLAGS:
-h, --help help for pool
Expand Down
25 changes: 25 additions & 0 deletions docs/commands/k8s.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ This API allows you to manage Kubernetes Kapsule and Kosmos clusters.
- [Replace a Node in a Cluster](#replace-a-node-in-a-cluster)
- [Wait for a node to reach a stable state](#wait-for-a-node-to-reach-a-stable-state)
- [Kapsule pool management commands](#kapsule-pool-management-commands)
- [Add an external node to a Kosmos Pool](#add-an-external-node-to-a-kosmos-pool)
- [Create a new Pool in a Cluster](#create-a-new-pool-in-a-cluster)
- [Delete a Pool in a Cluster](#delete-a-pool-in-a-cluster)
- [Get a Pool in a Cluster](#get-a-pool-in-a-cluster)
Expand Down Expand Up @@ -961,6 +962,30 @@ A pool is a set of identical nodes
A pool has a name, a size (its desired number of nodes), node number limits (min, max), and a Scaleway Instance type. Changing those limits increases/decreases the size of a pool. As a result and depending on its load, the pool will grow or shrink within those limits when autoscaling is enabled.


### Add an external node to a Kosmos Pool

Add an external node to a Kosmos Pool.
This will connect via SSH to the node, download the multicloud configuration script and run it with sudo privileges.
Keep in mind that your external node needs to have wget in order to download the script.

**Usage:**

```
scw k8s pool add-external-node [arg=value ...]
```


**Args:**

| Name | | Description |
|------|---|-------------|
| node-ip | Required | IP address of the external node |
| pool-id | Required | ID of the Pool the node should be added to |
| username | Default: `root` | Username used for the SSH connection |
| region | Default: `fr-par` | Region to target. If none is passed will use default region from the config |



### Create a new Pool in a Cluster

Create a new pool in a specific Kubernetes cluster.
Expand Down
1 change: 1 addition & 0 deletions internal/namespaces/k8s/v1/custom.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ func GetCommands() *core.Commands {
k8sClusterWaitCommand(),
k8sNodeWaitCommand(),
k8sPoolWaitCommand(),
k8sPoolAddExternalNodeCommand(),
))

human.RegisterMarshalerFunc(k8s.Version{}, versionMarshalerFunc)
Expand Down
112 changes: 112 additions & 0 deletions internal/namespaces/k8s/v1/custom_pool.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,15 @@ import (
"errors"
"fmt"
"net/http"
"os/exec"
"reflect"
"strings"
"time"

"github.com/fatih/color"
"github.com/scaleway/scaleway-cli/v2/core"
"github.com/scaleway/scaleway-cli/v2/core/human"
"github.com/scaleway/scaleway-cli/v2/internal/interactive"
k8s "github.com/scaleway/scaleway-sdk-go/api/k8s/v1"
"github.com/scaleway/scaleway-sdk-go/scw"
)
Expand Down Expand Up @@ -137,3 +140,112 @@ func k8sPoolWaitCommand() *core.Command {
},
}
}

type k8sPoolAddExternalNodeRequest struct {
NodeIP string
PoolID string
Username string
Region scw.Region
}

func k8sPoolAddExternalNodeCommand() *core.Command {
return &core.Command{
Short: `Add an external node to a Kosmos Pool`,
Long: `Add an external node to a Kosmos Pool.
This will connect via SSH to the node, download the multicloud configuration script and run it with sudo privileges.
Keep in mind that your external node needs to have wget in order to download the script.`,
Namespace: "k8s",
Resource: "pool",
Verb: "add-external-node",
ArgsType: reflect.TypeOf(k8sPoolAddExternalNodeRequest{}),
ArgSpecs: core.ArgSpecs{
{
Name: "node-ip",
Short: `IP address of the external node`,
Required: true,
},
{
Name: "pool-id",
Short: `ID of the Pool the node should be added to`,
Required: true,
},
{
Name: "username",
Short: "Username used for the SSH connection",
Default: core.DefaultValueSetter("root"),
},
core.RegionArgSpec(),
},
Run: func(ctx context.Context, argsI interface{}) (i interface{}, err error) {
args := argsI.(*k8sPoolAddExternalNodeRequest)
sshCommonArgs := []string{
args.NodeIP,
"-t",
"-l", args.Username,
}

// Set POOL_ID and REGION in the node init script and copy it to the remote
homeDir := "/root"
if args.Username != "root" {
homeDir = "/home/" + args.Username
}
nodeInitScript := buildNodeInitScript(args.PoolID, args.Region)
copyScriptArgs := []string{
"cat", "<<", "EOF",
">", homeDir + "/init_kosmos_node.sh",
"\n",
}
copyScriptArgs = append(copyScriptArgs, strings.Split(nodeInitScript, " \n")...)
if err = execSSHCommand(ctx, append(sshCommonArgs, copyScriptArgs...), true); err != nil {
return nil, err
}
chmodArgs := []string{"chmod", "+x", homeDir + "/init_kosmos_node.sh"}
if err = execSSHCommand(ctx, append(sshCommonArgs, chmodArgs...), true); err != nil {
return nil, err
}

// Execute the script with SCW_SECRET_KEY set
client := core.ExtractClient(ctx)
secretKey, _ := client.GetSecretKey()
execScriptArgs := []string{
"", // Adding a space to prevent the command from being logged in history as it shows the secret key
"SCW_SECRET_KEY=" + secretKey,
"./init_kosmos_node.sh",
}
if err = execSSHCommand(ctx, append(sshCommonArgs, execScriptArgs...), false); err != nil {
return nil, err
}

return &core.SuccessResult{Empty: true}, nil
},
}
}

func execSSHCommand(ctx context.Context, args []string, printSeparator bool) error {
remoteCmd := exec.Command("ssh", args...)
_, _ = interactive.Println(remoteCmd)

exitCode, err := core.ExecCmd(ctx, remoteCmd)
if err != nil {
return err
}
if exitCode != 0 {
return fmt.Errorf("ssh command failed with exit code %d", exitCode)
}
if printSeparator {
_, _ = interactive.Println("-----")
}

return nil
}

func buildNodeInitScript(poolID string, region scw.Region) string {
return fmt.Sprintf(`#!/usr/bin/env sh

set -e
wget https://scwcontainermulticloud.s3.fr-par.scw.cloud/node-agent_linux_amd64 --no-verbose
chmod +x node-agent_linux_amd64
export POOL_ID=%s POOL_REGION=%s SCW_SECRET_KEY=\$SCW_SECRET_KEY
sudo -E ./node-agent_linux_amd64 -loglevel 0 -no-controller
EOF`, poolID, region.String())
}
Loading