diff --git a/README.md b/README.md index 20707391..89a042cd 100644 --- a/README.md +++ b/README.md @@ -27,6 +27,7 @@ Whether you don't want to overload your Raspberry Pi, or your QA environment is - [Docker Swarm](#docker-swarm) - [Podman](#podman) - [Kubernetes](#kubernetes) + - [Proxmox LXC](#proxmox-lxc) - [Usage with Reverse Proxies](#usage-with-reverse-proxies) - [Apache APISIX](#apache-apisix) - [Caddy](#caddy) @@ -247,8 +248,8 @@ sablier --configFile=path/to/myconfigfile.yml ```yaml provider: - # Provider to use to manage containers (docker, swarm, kubernetes) - name: docker + # Provider to use to manage containers (docker, swarm, kubernetes, podman, proxmox_lxc) + name: docker server: # The server port to use port: 10000 @@ -390,6 +391,21 @@ Sablier provides native Kubernetes support for managing deployments, scaling wor 📚 **[Full Documentation](https://sablierapp.dev/#/providers/kubernetes)** +--- + +### Proxmox LXC + +Proxmox + +Sablier supports Proxmox VE for managing LXC containers on demand via the Proxmox API. + +**Features:** +- Connects to the Proxmox VE API with token authentication +- Starts/Stops LXC containers +- Discovers containers by `sablier` tag + +📚 **[Full Documentation](https://sablierapp.dev/#/providers/proxmox_lxc)** + ## Usage with Reverse Proxies Sablier is an API server that manages workload lifecycle. To automatically wake up workloads when users access your services, you can integrate Sablier with reverse proxy plugins. diff --git a/docs/README.md b/docs/README.md index fee4ed02..b823d4a1 100644 --- a/docs/README.md +++ b/docs/README.md @@ -17,7 +17,7 @@ This allows you to start your containers on demand and shut them down automatica Throughout this documentation, we use these terms to remain provider-agnostic: - **Session**: A session is a set of **instances** -- **Instance**: An instance is either a Docker container, Docker Swarm service, Kubernetes deployment, or Kubernetes StatefulSet +- **Instance**: An instance is either a Docker container, Docker Swarm service, Kubernetes deployment, Kubernetes StatefulSet, or Proxmox LXC container ## Credits diff --git a/docs/_sidebar.md b/docs/_sidebar.md index 09aa85a6..6b77fb45 100644 --- a/docs/_sidebar.md +++ b/docs/_sidebar.md @@ -11,6 +11,7 @@ - [Docker Swarm](/providers/docker_swarm) - [Kubernetes](/providers/kubernetes) - [Podman](/providers/podman) + - [Proxmox LXC](/providers/proxmox_lxc) - **Reverse Proxy Plugins** - [Overview](/plugins/overview) - [Apache APISIX](/plugins/apacheapisix) diff --git a/docs/assets/img/proxmox.png b/docs/assets/img/proxmox.png new file mode 100644 index 00000000..f6619f72 Binary files /dev/null and b/docs/assets/img/proxmox.png differ diff --git a/docs/configuration.md b/docs/configuration.md index 4e686ba2..615b1952 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -27,7 +27,7 @@ sablier --configFile=path/to/myconfigfile.yml ```yaml provider: - # Provider to use to manage containers (docker, swarm, kubernetes) + # Provider to use to manage containers (docker, swarm, kubernetes, podman, proxmox_lxc) name: docker docker: # Strategy to use for stopping Docker containers: stop or pause (default: stop) @@ -117,7 +117,7 @@ sablier start --strategy.dynamic.custom-themes-path /my/path ``` -h, --help help for start --provider.docker.strategy string Strategy to use to stop docker containers (stop or pause) (default "stop") - --provider.name string Provider to use to manage containers [docker swarm kubernetes] (default "docker") + --provider.name string Provider to use to manage containers [docker swarm kubernetes podman proxmox_lxc] (default "docker") --server.base-path string The base path for the API (default "/") --server.port int The server port to use (default 10000) --sessions.default-duration duration The default session duration (default 5m0s) diff --git a/docs/providers/overview.md b/docs/providers/overview.md index c5d7069c..08efe9e8 100644 --- a/docs/providers/overview.md +++ b/docs/providers/overview.md @@ -18,6 +18,7 @@ A Provider typically has the following capabilities: | [Docker Swarm](docker_swarm) | `docker_swarm` or `swarm` | Scale down to zero and up **services** on demand | | [Kubernetes](kubernetes) | `kubernetes` | Scale down and up **deployments** and **statefulsets** on demand | | [Podman](podman) | `podman` | Stop and start **containers** on demand | +| [Proxmox LXC](proxmox_lxc) | `proxmox_lxc` | Stop and start **LXC containers** on demand via Proxmox VE API | *Your Provider is not on the list? [Open an issue to request the missing provider here!](https://github.com/sablierapp/sablier/issues/new?assignees=&labels=enhancement%2C+provider&projects=&template=instance-provider-request.md&title=Add+%60%5BPROVIDER%5D%60+provider)* diff --git a/docs/providers/proxmox_lxc.md b/docs/providers/proxmox_lxc.md new file mode 100644 index 00000000..03bd47db --- /dev/null +++ b/docs/providers/proxmox_lxc.md @@ -0,0 +1,123 @@ +# Proxmox LXC + +The Proxmox LXC provider communicates with the Proxmox VE API to start and stop LXC containers on demand. + +## Use the Proxmox LXC provider + +In order to use the Proxmox LXC provider you can configure the [provider.name](../configuration) property. + + + +#### **File (YAML)** + +```yaml +provider: + name: proxmox_lxc + proxmox-lxc: + url: "https://proxmox.local:8006/api2/json" + token-id: "root@pam!sablier" + token-secret: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" + tls-insecure: false +``` + +#### **CLI** + +```bash +sablier start \ + --provider.name=proxmox_lxc \ + --provider.proxmox-lxc.url=https://proxmox.local:8006/api2/json \ + --provider.proxmox-lxc.token-id=root@pam!sablier \ + --provider.proxmox-lxc.token-secret=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx +``` + +#### **Environment Variable** + +```bash +SABLIER_PROVIDER_NAME=proxmox_lxc +SABLIER_PROVIDER_PROXMOX_LXC_URL=https://proxmox.local:8006/api2/json +SABLIER_PROVIDER_PROXMOX_LXC_TOKEN_ID=root@pam!sablier +SABLIER_PROVIDER_PROXMOX_LXC_TOKEN_SECRET=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx +SABLIER_PROVIDER_PROXMOX_LXC_TLS_INSECURE=false +``` + + + +## Configuration + +| Property | CLI Flag | Environment Variable | Default | Description | +|---|---|---|---|---| +| `url` | `--provider.proxmox-lxc.url` | `SABLIER_PROVIDER_PROXMOX_LXC_URL` | *(required)* | Proxmox VE API URL | +| `token-id` | `--provider.proxmox-lxc.token-id` | `SABLIER_PROVIDER_PROXMOX_LXC_TOKEN_ID` | *(required)* | API token ID (e.g. `root@pam!sablier`) | +| `token-secret` | `--provider.proxmox-lxc.token-secret` | `SABLIER_PROVIDER_PROXMOX_LXC_TOKEN_SECRET` | *(required)* | API token secret | +| `tls-insecure` | `--provider.proxmox-lxc.tls-insecure` | `SABLIER_PROVIDER_PROXMOX_LXC_TLS_INSECURE` | `false` | Skip TLS certificate verification (for self-signed certs) | + +## Create a Proxmox API Token + +1. In the Proxmox web UI, go to **Datacenter > Permissions > API Tokens** +2. Click **Add** and create a token for a user (e.g. `root@pam`) +3. Uncheck **Privilege Separation** so the token inherits the user's permissions +4. Note the **Token ID** (e.g. `root@pam!sablier`) and **Secret** + +The token needs the following permissions on the LXC containers: +- `VM.PowerMgmt` — to start and stop containers +- `VM.Audit` — to read container status and configuration + +## Register containers + +For Sablier to work, it needs to know which LXC containers to start and stop. + +You have to register your containers by opting-in with **Proxmox tags**. + +```yaml +arch: amd64 +cores: 2 +hostname: whoami +memory: 4096 +net0: name=eth0,bridge=vmbr0,hwaddr=BC:24:11:81:7C:C4,ip=dhcp,type=veth +ostype: debian +rootfs: local-lvm:vm-100-disk-0,size=8G +swap: 512 +tags: sablier;sablier-group-mygroup +unprivileged: 1 +``` + +### Add tags via the CLI + +```bash +pct set 100 -tags "sablier;sablier-group-mygroup" +``` + +### Add tags via the Web UI + +In the Proxmox web UI, select a container and click the **pencil icon** next to the tags in the toolbar (next to the container name) to edit tags. + +### Tags reference + +| Tag | Description | +|---|---| +| `sablier` | **Required.** Marks the container as managed by Sablier. | +| `sablier-group-` | Optional. Assigns the container to a group. Defaults to `default` if not specified. | + +## Instance naming + +Sablier uses the LXC container **hostname** as the instance name. You can also reference containers by their **VMID** (e.g. `100`) or by **node/VMID** format (e.g. `pve1/100`). + +!> Hostnames must be unique among Sablier-managed containers. If duplicate hostnames are detected, Sablier will return an error. + +## Multi-node support + +The Proxmox LXC provider automatically discovers all nodes in the cluster and scans for tagged containers across all of them. No additional configuration is required for multi-node setups. + +## How does Sablier know when a container is ready? + +Sablier checks the LXC container status reported by Proxmox. Additionally, for `running` containers, Sablier verifies that at least one non-loopback network interface has an IP address assigned before reporting the container as ready. + +| Proxmox Status | Sablier Status | +|---|---| +| `running` (with IP) | Ready | +| `running` (no IP yet) | Not Ready | +| `stopped` | Not Ready | +| `stopped` (after failed start) | Unrecoverable | +| Other | Unrecoverable | + +?> When a start task fails (e.g. `startup for container '100' failed`), Sablier marks the instance as **Unrecoverable** instead of retrying indefinitely. The failed-start state is cleared automatically after a short fixed TTL (currently about 30 seconds), allowing a new start attempt on a subsequent request even if the session is still active. diff --git a/go.mod b/go.mod index 4d49b4d8..905d1156 100644 --- a/go.mod +++ b/go.mod @@ -6,6 +6,7 @@ require ( github.com/gin-gonic/gin v1.12.0 github.com/google/go-cmp v0.7.0 github.com/lmittmann/tint v1.1.3 + github.com/luthermonson/go-proxmox v0.4.0 github.com/moby/moby/api v1.54.2 github.com/moby/moby/client v0.4.1 github.com/neilotoole/slogt v1.1.0 @@ -33,6 +34,7 @@ require ( dario.cat/mergo v1.0.2 // indirect github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c // indirect github.com/Microsoft/go-winio v0.6.2 // indirect + github.com/buger/goterm v1.0.4 // indirect github.com/bytedance/gopkg v0.1.3 // indirect github.com/bytedance/sonic v1.15.0 // indirect github.com/bytedance/sonic/loader v0.5.0 // indirect @@ -45,7 +47,9 @@ require ( github.com/containerd/platforms v1.0.0-rc.1 // indirect github.com/cpuguy83/dockercfg v0.3.2 // indirect github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect + github.com/diskfs/go-diskfs v1.7.0 // indirect github.com/distribution/reference v0.6.0 // indirect + github.com/djherbis/times v1.6.0 // indirect github.com/docker/go-connections v0.7.0 // indirect github.com/docker/go-units v0.5.0 // indirect github.com/ebitengine/purego v0.10.0 // indirect @@ -69,13 +73,16 @@ require ( github.com/goccy/go-yaml v1.19.2 // indirect github.com/google/gnostic-models v0.7.0 // indirect github.com/google/uuid v1.6.0 // indirect + github.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674 // indirect github.com/inconshreveable/mousetrap v1.1.0 // indirect + github.com/jinzhu/copier v0.3.4 // indirect github.com/josharian/intern v1.0.0 // indirect github.com/json-iterator/go v1.1.12 // indirect github.com/klauspost/compress v1.18.5 // indirect github.com/klauspost/cpuid/v2 v2.3.0 // indirect github.com/leodido/go-urn v1.4.0 // indirect github.com/lufia/plan9stats v0.0.0-20240909124753-873cd0166683 // indirect + github.com/magefile/mage v1.14.0 // indirect github.com/magiconair/properties v1.8.10 // indirect github.com/mailru/easyjson v0.7.7 // indirect github.com/mattn/go-isatty v0.0.20 // indirect diff --git a/go.sum b/go.sum index 5d7ab041..03ef8175 100644 --- a/go.sum +++ b/go.sum @@ -6,6 +6,10 @@ github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c h1:udKWzYgxTojEK github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c/go.mod h1:xomTg63KZ2rFqZQzSB4Vz2SUXa1BpHTVz9L5PTmPC4E= github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY= github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU= +github.com/anchore/go-lzo v0.1.0 h1:NgAacnzqPeGH49Ky19QKLBZEuFRqtTG9cdaucc3Vncs= +github.com/anchore/go-lzo v0.1.0/go.mod h1:3kLx0bve2oN1iDwgM1U5zGku1Tfbdb0No5qp1eL1fIk= +github.com/buger/goterm v1.0.4 h1:Z9YvGmOih81P0FbVtEYTFF6YsSgxSUKEhf/f9bTMXbY= +github.com/buger/goterm v1.0.4/go.mod h1:HiFWV3xnkolgrBV3mY8m0X0Pumt4zg4QhbdOzQtB8tE= github.com/bytedance/gopkg v0.1.3 h1:TPBSwH8RsouGCBcMBktLt1AymVo2TVsBVCY4b6TnZ/M= github.com/bytedance/gopkg v0.1.3/go.mod h1:576VvJ+eJgyCzdjS+c4+77QF3p7ubbtiKARP3TxducM= github.com/bytedance/sonic v1.15.0 h1:/PXeWFaR5ElNcVE84U0dOHjiMHQOwNIx3K4ymzh/uSE= @@ -36,14 +40,20 @@ github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSs github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM= github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/diskfs/go-diskfs v1.7.0 h1:vonWmt5CMowXwUc79jWyGrf2DIMeoOjkLlMnQYGVOs8= +github.com/diskfs/go-diskfs v1.7.0/go.mod h1:LhQyXqOugWFRahYUSw47NyZJPezFzB9UELwhpszLP/k= github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5QvfrDyIgxBk= github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E= +github.com/djherbis/times v1.6.0 h1:w2ctJ92J8fBvWPxugmXIv7Nz7Q3iDMKNx9v5ocVH20c= +github.com/djherbis/times v1.6.0/go.mod h1:gOHeRAz2h+VJNZ5Gmc/o7iD9k4wW7NMVqieYCY99oc0= github.com/docker/go-connections v0.7.0 h1:6SsRfJddP22WMrCkj19x9WKjEDTB+ahsdiGYf0mN39c= github.com/docker/go-connections v0.7.0/go.mod h1:no1qkHdjq7kLMGUXYAduOhYPSJxxvgWBh7ogVvptn3Q= github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4= github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk= github.com/ebitengine/purego v0.10.0 h1:QIw4xfpWT6GWTzaW5XEKy3HXoqrJGx1ijYHzTF0/ISU= github.com/ebitengine/purego v0.10.0/go.mod h1:iIjxzd6CiRiOG0UyXP+V1+jWqUXVjPKLAI0mRfJZTmQ= +github.com/elliotwutingfeng/asciiset v0.0.0-20230602022725-51bbb787efab h1:h1UgjJdAAhj+uPL68n7XASS6bU+07ZX1WJvVS2eyoeY= +github.com/elliotwutingfeng/asciiset v0.0.0-20230602022725-51bbb787efab/go.mod h1:GLo/8fDswSAniFG+BFIaiSPcK610jyzgEhWYPQwuQdw= github.com/emicklei/go-restful/v3 v3.13.0 h1:C4Bl2xDndpU6nJ4bc1jXd+uTmYPVUwkD6bFY/oTyCes= github.com/emicklei/go-restful/v3 v3.13.0/go.mod h1:6n3XBCmQQb25CM2LCACGz8ukIrRry+4bhvbpWn3mrbc= github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg= @@ -88,6 +98,8 @@ github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91 github.com/go-playground/validator/v10 v10.30.1 h1:f3zDSN/zOma+w6+1Wswgd9fLkdwy06ntQJp0BBvFG0w= github.com/go-playground/validator/v10 v10.30.1/go.mod h1:oSuBIQzuJxL//3MelwSLD5hc2Tu889bF0Idm9Dg26cM= github.com/go-task/slim-sprig v0.0.0-20210107165309-348f09dbbbc0/go.mod h1:fyg7847qk6SyHyPtNmDHnmrv/HOrqktSC+C9fM+CJOE= +github.com/go-test/deep v1.0.8 h1:TDsG77qcSprGbC6vTN8OuXp5g+J+b5Pcguhf7Zt61VM= +github.com/go-test/deep v1.0.8/go.mod h1:5C2ZWiW0ErCdrYzpqxLbTX7MG14M9iiw8DgHncVwcsE= github.com/go-viper/mapstructure/v2 v2.4.0 h1:EBsztssimR/CONLSZZ04E8qAkxNYq4Qp9LvH92wZUgs= github.com/go-viper/mapstructure/v2 v2.4.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM= github.com/goccy/go-json v0.10.5 h1:Fq85nIqj+gXn/S5ahsiTlK3TmC85qgirsdTP/+DeaC4= @@ -112,9 +124,17 @@ github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674 h1:JeSE6pjso5THxAzdVpqr6/geYxZytqFMBCOtn/ujyeo= +github.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674/go.mod h1:r4w70xmWCQKmi1ONH4KIaBptdivuRPyosB9RmPlGEwA= +github.com/h2non/gock v1.2.0 h1:K6ol8rfrRkUOefooBC8elXoaNGYkpp7y2qcxGG6BzUE= +github.com/h2non/gock v1.2.0/go.mod h1:tNhoxHYW2W42cYkYb1WqzdbYIieALC99kpYr7rH/BQk= +github.com/h2non/parth v0.0.0-20190131123155-b4df798d6542 h1:2VTzZjLZBgl62/EtslCrtky5vbi9dd7HrQPQIx6wqiw= +github.com/h2non/parth v0.0.0-20190131123155-b4df798d6542/go.mod h1:Ow0tF8D4Kplbc8s8sSb3V2oUCygFHVp8gC3Dn6U4MNI= github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU= github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8= github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw= +github.com/jinzhu/copier v0.3.4 h1:mfU6jI9PtCeUjkjQ322dlff9ELjGDu975C2p/nrubVI= +github.com/jinzhu/copier v0.3.4/go.mod h1:DfbEm0FYsaqBcKcFuvmOZb218JkPGtvSHsKg8S8hyyg= github.com/josharian/intern v1.0.0 h1:vlS4z54oSdjm0bgjRigI+G1HpF+tI+9rE5LLzOg8HmY= github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y= github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM= @@ -136,6 +156,10 @@ github.com/lmittmann/tint v1.1.3 h1:Hv4EaHWXQr+GTFnOU4VKf8UvAtZgn0VuKT+G0wFlO3I= github.com/lmittmann/tint v1.1.3/go.mod h1:HIS3gSy7qNwGCj+5oRjAutErFBl4BzdQP6cJZ0NfMwE= github.com/lufia/plan9stats v0.0.0-20240909124753-873cd0166683 h1:7UMa6KCCMjZEMDtTVdcGu0B1GmmC7QJKiCCjyTAWQy0= github.com/lufia/plan9stats v0.0.0-20240909124753-873cd0166683/go.mod h1:ilwx/Dta8jXAgpFYFvSWEMwxmbWXyiUHkd5FwyKhb5k= +github.com/luthermonson/go-proxmox v0.4.0 h1:LKXpG9d64zTaQF79wV0kfOnnSwIcdG39m7sc4ga+XZs= +github.com/luthermonson/go-proxmox v0.4.0/go.mod h1:U6dAkJ+iiwaeb1g/LMWpWuWN4nmvWeXhmoMuYJMumS4= +github.com/magefile/mage v1.14.0 h1:6QDX3g6z1YvJ4olPhT1wksUcSa/V0a1B+pJb73fBjyo= +github.com/magefile/mage v1.14.0/go.mod h1:z5UZb/iS3GoOSn0JgWuiw7dxlurVYTu+/jHXqQg881A= github.com/magiconair/properties v1.8.10 h1:s31yESBquKXCV9a/ScB3ESkOjUYYv+X0rg8SYxI99mE= github.com/magiconair/properties v1.8.10/go.mod h1:Dhd985XPs7jluiymwWYZ0G4Z61jb3vdS329zhj2hYo0= github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0= @@ -190,8 +214,12 @@ github.com/opencontainers/image-spec v1.1.1 h1:y0fUlFfIZhPF1W537XOLg0/fcx6zcHCJw github.com/opencontainers/image-spec v1.1.1/go.mod h1:qpqAh3Dmcf36wStyyWU+kCeDgrGnAve2nCC8+7h8Q0M= github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4= github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY= +github.com/pierrec/lz4/v4 v4.1.17 h1:kV4Ip+/hUBC+8T6+2EgburRtkE9ef4nbY3f4dFhGjMc= +github.com/pierrec/lz4/v4 v4.1.17/go.mod h1:gZWDp/Ze/IJXGXf23ltt2EXimqmTUXEy0GFuRQyBid4= github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= +github.com/pkg/xattr v0.4.9 h1:5883YPCtkSd8LFbs13nXplj9g9tlrwoJRjgpgMu1/fE= +github.com/pkg/xattr v0.4.9/go.mod h1:di8WF84zAKk8jzR1UBTEWh9AUlIZZ7M/JNt8e9B6ktU= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U= github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= @@ -260,6 +288,8 @@ github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08= github.com/ugorji/go/codec v1.3.1 h1:waO7eEiFDwidsBN6agj1vJQ4AG7lh2yqXyOXqhgQuyY= github.com/ugorji/go/codec v1.3.1/go.mod h1:pRBVtBSKl77K30Bv8R2P+cLSGaTtex6fsA2Wjqmfxj4= +github.com/ulikunitz/xz v0.5.11 h1:kpFauv27b6ynzBNT/Xy+1k+fK4WswhN/6PN5WhFAGw8= +github.com/ulikunitz/xz v0.5.11/go.mod h1:nbz6k7qbPmH4IRqmfOplQw/tblSgqTqBwxkY0oWt/14= github.com/valkey-io/valkey-go v1.0.74 h1:NqtBHzjybz+is+c71hsyZP7hoE5lwCHQX026me0Vb08= github.com/valkey-io/valkey-go v1.0.74/go.mod h1:VGhZ6fs68Qrn2+OhH+6waZH27bjpgQOiLyUQyXuYK5k= github.com/x448/float16 v0.8.4 h1:qLwI1I70+NjRFUR3zs1JPUCgaCXSh3SW62uAKT1mSBM= @@ -324,7 +354,9 @@ golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7w golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20201204225414-ed752295db88/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210112080510-489259a85091/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210331175145-43e1dd70ce54/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210616094352-59db8d763f22/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220615213510-4f61da869c0c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220908164124-27713097b956/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= diff --git a/pkg/config/provider.go b/pkg/config/provider.go index b36459c5..893804f6 100644 --- a/pkg/config/provider.go +++ b/pkg/config/provider.go @@ -7,12 +7,13 @@ import ( // Provider holds the provider configurations type Provider struct { // The provider name to use - // It can be either docker, swarm or kubernetes. Defaults to "docker" + // It can be either docker, swarm, kubernetes, podman or proxmox_lxc. Defaults to "docker" Name string `mapstructure:"NAME" yaml:"name,omitempty" default:"docker"` AutoStopOnStartup bool `yaml:"auto-stop-on-startup,omitempty" default:"true"` Kubernetes Kubernetes Podman Podman Docker Docker + ProxmoxLXC ProxmoxLXC `mapstructure:"PROXMOX_LXC" yaml:"proxmox-lxc,omitempty"` } type Kubernetes struct { @@ -39,7 +40,19 @@ type Docker struct { Strategy string `mapstructure:"STRATEGY" yaml:"strategy,omitempty" default:"stop"` } -var providers = []string{"docker", "docker_swarm", "swarm", "kubernetes", "podman"} +// ProxmoxLXC holds the Proxmox VE LXC provider configuration +type ProxmoxLXC struct { + // URL is the Proxmox VE API endpoint (e.g. "https://proxmox:8006/api2/json") + URL string `mapstructure:"URL" yaml:"url,omitempty"` + // TokenID is the API token identifier (e.g. "root@pam!sablier") + TokenID string `mapstructure:"TOKEN_ID" yaml:"token-id,omitempty"` + // TokenSecret is the API token secret UUID + TokenSecret string `mapstructure:"TOKEN_SECRET" yaml:"token-secret,omitempty"` + // TLSInsecure skips TLS certificate verification (useful for self-signed certificates) + TLSInsecure bool `mapstructure:"TLS_INSECURE" yaml:"tls-insecure,omitempty" default:"false"` +} + +var providers = []string{"docker", "docker_swarm", "swarm", "kubernetes", "podman", "proxmox_lxc"} var dockerStrategies = []string{"stop", "pause"} func NewProviderConfig() Provider { @@ -57,6 +70,7 @@ func NewProviderConfig() Provider { Docker: Docker{ Strategy: "stop", }, + ProxmoxLXC: ProxmoxLXC{}, } } @@ -69,6 +83,12 @@ func (provider Provider) IsValid() error { return err } } + // Validate Proxmox LXC-specific settings + if p == "proxmox_lxc" { + if err := provider.ProxmoxLXC.IsValid(); err != nil { + return err + } + } return nil } } @@ -84,6 +104,19 @@ func (docker Docker) IsValid() error { return fmt.Errorf("unrecognized docker strategy %s. strategies available: %v", docker.Strategy, dockerStrategies) } +func (p ProxmoxLXC) IsValid() error { + if p.URL == "" { + return fmt.Errorf("proxmox_lxc provider requires a URL") + } + if p.TokenID == "" { + return fmt.Errorf("proxmox_lxc provider requires a token ID") + } + if p.TokenSecret == "" { + return fmt.Errorf("proxmox_lxc provider requires a token secret") + } + return nil +} + func GetProviders() []string { return providers } diff --git a/pkg/config/provider_test.go b/pkg/config/provider_test.go index e5ed14d5..4c664b96 100644 --- a/pkg/config/provider_test.go +++ b/pkg/config/provider_test.go @@ -71,12 +71,53 @@ func TestProvider_IsValid(t *testing.T) { }, wantErr: nil, }, + { + name: "valid proxmox_lxc provider", + provider: Provider{ + Name: "proxmox_lxc", + ProxmoxLXC: ProxmoxLXC{ + URL: "https://proxmox:8006/api2/json", + TokenID: "root@pam!sablier", + TokenSecret: "secret", + }, + }, + wantErr: nil, + }, + { + name: "proxmox_lxc provider missing url", + provider: Provider{ + Name: "proxmox_lxc", + ProxmoxLXC: ProxmoxLXC{}, + }, + wantErr: fmt.Errorf("proxmox_lxc provider requires a URL"), + }, + { + name: "proxmox_lxc provider missing token id", + provider: Provider{ + Name: "proxmox_lxc", + ProxmoxLXC: ProxmoxLXC{ + URL: "https://proxmox:8006/api2/json", + }, + }, + wantErr: fmt.Errorf("proxmox_lxc provider requires a token ID"), + }, + { + name: "proxmox_lxc provider missing token secret", + provider: Provider{ + Name: "proxmox_lxc", + ProxmoxLXC: ProxmoxLXC{ + URL: "https://proxmox:8006/api2/json", + TokenID: "root@pam!sablier", + }, + }, + wantErr: fmt.Errorf("proxmox_lxc provider requires a token secret"), + }, { name: "invalid provider name", provider: Provider{ Name: "invalid", }, - wantErr: fmt.Errorf("unrecognized provider invalid. providers available: [docker docker_swarm swarm kubernetes podman]"), + wantErr: fmt.Errorf("unrecognized provider invalid. providers available: [docker docker_swarm swarm kubernetes podman proxmox_lxc]"), }, } diff --git a/pkg/provider/proxmoxlxc/container_inspect.go b/pkg/provider/proxmoxlxc/container_inspect.go new file mode 100644 index 00000000..a0a4e142 --- /dev/null +++ b/pkg/provider/proxmoxlxc/container_inspect.go @@ -0,0 +1,208 @@ +package proxmoxlxc + +import ( + "context" + "fmt" + "log/slog" + "time" + + proxmox "github.com/luthermonson/go-proxmox" + "github.com/sablierapp/sablier/pkg/sablier" +) + +func (p *Provider) InstanceInspect(ctx context.Context, name string) (sablier.InstanceInfo, error) { + ref, err := p.resolve(ctx, name) + if err != nil { + return sablier.InstanceInfo{}, fmt.Errorf("cannot resolve instance %q: %w", name, err) + } + + // Check if there is a pending start task for this instance. + p.mu.Lock() + pt, hasPending := p.pendingTasks[name] + p.mu.Unlock() + + if hasPending { + info, done := p.checkPendingTask(ctx, name, ref, pt) + if !done { + return info, nil + } + // If the failed task TTL expired, attempt a fresh start so the caller + // (Sablier core) doesn't need to call InstanceStart — its session still + // exists with not-ready status, so only InstanceInspect will be called. + if !pt.failedAt.IsZero() { + if err := p.InstanceStart(ctx, name); err != nil { + p.l.WarnContext(ctx, "retry start after failed task expiry failed", + slog.String("name", ref.name), + slog.Any("error", err), + ) + } else { + return sablier.InstanceInfo{ + Name: ref.name, + CurrentReplicas: 0, + DesiredReplicas: p.desiredReplicas, + Status: sablier.InstanceStatusStarting, + }, nil + } + } + // Task completed successfully — fall through to normal status check. + } + + ct, err := p.getContainer(ctx, ref) + if err != nil { + return sablier.InstanceInfo{}, err + } + + p.l.DebugContext(ctx, "container inspected", + slog.String("name", ref.name), + slog.Int("vmid", ref.vmid), + slog.String("node", ref.node), + slog.String("status", ct.Status), + ) + + switch ct.Status { + case "running": + // Check if the container's network interfaces have an IP address assigned. + // Proxmox reports "running" as soon as the LXC starts, but services inside + // may not be ready yet. Checking for a non-loopback interface with an IP + // is a lightweight readiness signal. + ifaces, err := ct.Interfaces(ctx) + if err != nil { + p.l.WarnContext(ctx, "cannot check container interfaces, assuming ready", + slog.String("name", ref.name), + slog.Any("error", err), + ) + return sablier.InstanceInfo{ + Name: ref.name, + CurrentReplicas: p.desiredReplicas, + DesiredReplicas: p.desiredReplicas, + Status: sablier.InstanceStatusReady, + }, nil + } + + if !hasNonLoopbackIP(ifaces) { + p.l.DebugContext(ctx, "container running but no network interface has an IP yet", + slog.String("name", ref.name), + ) + return sablier.InstanceInfo{ + Name: ref.name, + CurrentReplicas: 0, + DesiredReplicas: p.desiredReplicas, + Status: sablier.InstanceStatusStarting, + }, nil + } + + return sablier.InstanceInfo{ + Name: ref.name, + CurrentReplicas: p.desiredReplicas, + DesiredReplicas: p.desiredReplicas, + Status: sablier.InstanceStatusReady, + }, nil + case "stopped": + return sablier.InstanceInfo{ + Name: ref.name, + CurrentReplicas: 0, + DesiredReplicas: p.desiredReplicas, + Status: sablier.InstanceStatusStopped, + }, nil + default: + return sablier.InstanceInfo{ + Name: ref.name, + CurrentReplicas: 0, + DesiredReplicas: p.desiredReplicas, + Status: sablier.InstanceStatusError, + Message: fmt.Sprintf("container status %q not handled", ct.Status), + }, nil + } +} + +// checkPendingTask polls the Proxmox task status and returns the appropriate state. +// It returns (info, done) where done=true means the task completed (or the failure +// TTL expired) and the caller should fall through to the normal container status check. +func (p *Provider) checkPendingTask(ctx context.Context, name string, ref containerRef, pt *pendingTask) (sablier.InstanceInfo, bool) { + if err := pt.task.Ping(ctx); err != nil { + p.l.WarnContext(ctx, "cannot ping start task, assuming in progress", + slog.String("name", ref.name), + slog.Any("error", err), + ) + return sablier.InstanceInfo{ + Name: ref.name, + CurrentReplicas: 0, + DesiredReplicas: p.desiredReplicas, + Status: sablier.InstanceStatusStarting, + }, false + } + + if pt.task.IsRunning { + p.l.DebugContext(ctx, "start task still running", + slog.String("name", ref.name), + slog.String("upid", string(pt.task.UPID)), + ) + return sablier.InstanceInfo{ + Name: ref.name, + CurrentReplicas: 0, + DesiredReplicas: p.desiredReplicas, + Status: sablier.InstanceStatusStarting, + }, false + } + + if pt.task.IsFailed { + // Record when we first observed the failure. task.EndTime from the API + // is preferred, but copier.Copy in go-proxmox may zero it out (json:"-"), + // so failedAt is a reliable fallback. + if pt.failedAt.IsZero() { + if !pt.task.EndTime.IsZero() { + pt.failedAt = pt.task.EndTime + } else { + pt.failedAt = time.Now() + } + } + + // Keep returning Error until failedTaskTTL after the task finished. + // This gives the user time to see the error while polling, and automatically + // clears the entry so a fresh InstanceStart can be attempted later. + if time.Since(pt.failedAt) > failedTaskTTL { + p.mu.Lock() + delete(p.pendingTasks, name) + p.mu.Unlock() + p.l.InfoContext(ctx, "cleared expired failed start task", + slog.String("name", ref.name), + slog.Duration("since_failed", time.Since(pt.failedAt)), + ) + return sablier.InstanceInfo{}, true + } + msg := fmt.Sprintf("start task failed for container %q (VMID %d): %s", ref.name, ref.vmid, pt.task.ExitStatus) + p.l.WarnContext(ctx, msg) + return sablier.InstanceInfo{ + Name: ref.name, + CurrentReplicas: 0, + DesiredReplicas: p.desiredReplicas, + Status: sablier.InstanceStatusError, + Message: msg, + }, false + } + + // Task completed successfully — remove from pending. + p.mu.Lock() + delete(p.pendingTasks, name) + p.mu.Unlock() + + p.l.DebugContext(ctx, "start task completed successfully", + slog.String("name", ref.name), + slog.String("upid", string(pt.task.UPID)), + ) + // done=true: task succeeded, fall through to normal container status check. + return sablier.InstanceInfo{}, true +} + +// hasNonLoopbackIP returns true if any non-loopback interface has an IPv4 or IPv6 address. +func hasNonLoopbackIP(ifaces proxmox.ContainerInterfaces) bool { + for _, iface := range ifaces { + if iface.Name == "lo" { + continue + } + if iface.Inet != "" || iface.Inet6 != "" { + return true + } + } + return false +} diff --git a/pkg/provider/proxmoxlxc/container_inspect_test.go b/pkg/provider/proxmoxlxc/container_inspect_test.go new file mode 100644 index 00000000..d4e113ed --- /dev/null +++ b/pkg/provider/proxmoxlxc/container_inspect_test.go @@ -0,0 +1,124 @@ +package proxmoxlxc_test + +import ( + "testing" + + "github.com/neilotoole/slogt" + "github.com/sablierapp/sablier/pkg/provider/proxmoxlxc" + "github.com/sablierapp/sablier/pkg/sablier" + "gotest.tools/v3/assert" +) + +func TestProxmoxLXCProvider_InstanceInspect(t *testing.T) { + t.Parallel() + + tests := []struct { + name string + container proxmoxlxc.TestContainer + want sablier.InstanceInfo + }{ + { + name: "running container is ready", + container: proxmoxlxc.TestContainer{VMID: 100, Name: "web", Status: "running", Tags: "sablier", Node: "pve1"}, + want: sablier.InstanceInfo{ + Name: "web", + CurrentReplicas: 1, + DesiredReplicas: 1, + Status: sablier.InstanceStatusReady, + }, + }, + { + name: "stopped container is not ready", + container: proxmoxlxc.TestContainer{VMID: 101, Name: "db", Status: "stopped", Tags: "sablier", Node: "pve1"}, + want: sablier.InstanceInfo{ + Name: "db", + CurrentReplicas: 0, + DesiredReplicas: 1, + Status: sablier.InstanceStatusStopped, + }, + }, + { + name: "unknown status is unrecoverable", + container: proxmoxlxc.TestContainer{VMID: 102, Name: "backup", Status: "unknown", Tags: "sablier", Node: "pve1"}, + want: sablier.InstanceInfo{ + Name: "backup", + CurrentReplicas: 0, + DesiredReplicas: 1, + Status: sablier.InstanceStatusError, + Message: "container status \"unknown\" not handled", + }, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + + server := proxmoxlxc.MockServer(t, []string{"pve1"}, []proxmoxlxc.TestContainer{tt.container}) + defer server.Close() + + p, err := proxmoxlxc.New(t.Context(), proxmoxlxc.NewTestClient(server.URL), slogt.New(t)) + assert.NilError(t, err) + + got, err := p.InstanceInspect(t.Context(), tt.container.Name) + assert.NilError(t, err) + assert.DeepEqual(t, got, tt.want) + }) + } +} + +func TestProxmoxLXCProvider_InstanceInspect_ByVMID(t *testing.T) { + t.Parallel() + + server := proxmoxlxc.MockServer(t, []string{"pve1"}, []proxmoxlxc.TestContainer{ + {VMID: 100, Name: "web", Status: "running", Tags: "sablier", Node: "pve1"}, + }) + defer server.Close() + + p, err := proxmoxlxc.New(t.Context(), proxmoxlxc.NewTestClient(server.URL), slogt.New(t)) + assert.NilError(t, err) + + got, err := p.InstanceInspect(t.Context(), "100") + assert.NilError(t, err) + assert.DeepEqual(t, got, sablier.InstanceInfo{ + Name: "web", + CurrentReplicas: 1, + DesiredReplicas: 1, + Status: sablier.InstanceStatusReady, + }) +} + +func TestProxmoxLXCProvider_InstanceInspect_ByNodeVMID(t *testing.T) { + t.Parallel() + + server := proxmoxlxc.MockServer(t, []string{"pve1"}, []proxmoxlxc.TestContainer{ + {VMID: 100, Name: "web", Status: "running", Tags: "sablier", Node: "pve1"}, + }) + defer server.Close() + + p, err := proxmoxlxc.New(t.Context(), proxmoxlxc.NewTestClient(server.URL), slogt.New(t)) + assert.NilError(t, err) + + got, err := p.InstanceInspect(t.Context(), "pve1/100") + assert.NilError(t, err) + // node/vmid resolves to the hostname via scanContainers. + assert.DeepEqual(t, got, sablier.InstanceInfo{ + Name: "web", + CurrentReplicas: 1, + DesiredReplicas: 1, + Status: sablier.InstanceStatusReady, + }) +} + +func TestProxmoxLXCProvider_InstanceInspect_NotFound(t *testing.T) { + t.Parallel() + + server := proxmoxlxc.MockServer(t, []string{"pve1"}, []proxmoxlxc.TestContainer{}) + defer server.Close() + + p, err := proxmoxlxc.New(t.Context(), proxmoxlxc.NewTestClient(server.URL), slogt.New(t)) + assert.NilError(t, err) + + _, err = p.InstanceInspect(t.Context(), "nonexistent") + assert.ErrorContains(t, err, "not found") +} diff --git a/pkg/provider/proxmoxlxc/container_list.go b/pkg/provider/proxmoxlxc/container_list.go new file mode 100644 index 00000000..d56fe0e6 --- /dev/null +++ b/pkg/provider/proxmoxlxc/container_list.go @@ -0,0 +1,55 @@ +package proxmoxlxc + +import ( + "context" + "log/slog" + "sort" + + "github.com/sablierapp/sablier/pkg/provider" + "github.com/sablierapp/sablier/pkg/sablier" +) + +func (p *Provider) InstanceList(ctx context.Context, options provider.InstanceListOptions) ([]sablier.InstanceConfiguration, error) { + discovered, err := p.scanContainers(ctx) + if err != nil { + return nil, err + } + + p.l.DebugContext(ctx, "containers listed", slog.Int("count", len(discovered)), slog.Bool("all", options.All)) + + instances := make([]sablier.InstanceConfiguration, 0, len(discovered)) + for _, d := range discovered { + if !options.All && d.status != "running" { + continue + } + instances = append(instances, sablier.InstanceConfiguration{ + Name: d.ref.name, + Group: extractGroup(d.tags), + Enabled: "true", + }) + } + + return instances, nil +} + +func (p *Provider) InstanceGroups(ctx context.Context) (map[string][]string, error) { + discovered, err := p.scanContainers(ctx) + if err != nil { + return nil, err + } + + p.l.DebugContext(ctx, "containers listed for groups", slog.Int("count", len(discovered))) + + groups := make(map[string][]string) + for _, d := range discovered { + groupName := extractGroup(d.tags) + groups[groupName] = append(groups[groupName], d.ref.name) + } + + // Sort instance names within each group for stable ordering + for _, names := range groups { + sort.Strings(names) + } + + return groups, nil +} diff --git a/pkg/provider/proxmoxlxc/container_list_test.go b/pkg/provider/proxmoxlxc/container_list_test.go new file mode 100644 index 00000000..51235a8d --- /dev/null +++ b/pkg/provider/proxmoxlxc/container_list_test.go @@ -0,0 +1,120 @@ +package proxmoxlxc_test + +import ( + "sort" + "testing" + + "github.com/neilotoole/slogt" + "github.com/sablierapp/sablier/pkg/provider" + "github.com/sablierapp/sablier/pkg/provider/proxmoxlxc" + "github.com/sablierapp/sablier/pkg/sablier" + "gotest.tools/v3/assert" +) + +func TestProxmoxLXCProvider_InstanceList(t *testing.T) { + t.Parallel() + + server := proxmoxlxc.MockServer(t, []string{"pve1"}, []proxmoxlxc.TestContainer{ + {VMID: 100, Name: "web", Status: "running", Tags: "sablier;sablier-group-frontend", Node: "pve1"}, + {VMID: 101, Name: "db", Status: "stopped", Tags: "sablier", Node: "pve1"}, + {VMID: 102, Name: "unmanaged", Status: "running", Tags: "production", Node: "pve1"}, + }) + defer server.Close() + + p, err := proxmoxlxc.New(t.Context(), proxmoxlxc.NewTestClient(server.URL), slogt.New(t)) + assert.NilError(t, err) + + instances, err := p.InstanceList(t.Context(), provider.InstanceListOptions{All: true}) + assert.NilError(t, err) + + sort.Slice(instances, func(i, j int) bool { return instances[i].Name < instances[j].Name }) + assert.DeepEqual(t, instances, []sablier.InstanceConfiguration{ + {Name: "db", Group: "default", Enabled: "true"}, + {Name: "web", Group: "frontend", Enabled: "true"}, + }) +} + +func TestProxmoxLXCProvider_InstanceList_RunningOnly(t *testing.T) { + t.Parallel() + + server := proxmoxlxc.MockServer(t, []string{"pve1"}, []proxmoxlxc.TestContainer{ + {VMID: 100, Name: "web", Status: "running", Tags: "sablier;sablier-group-frontend", Node: "pve1"}, + {VMID: 101, Name: "db", Status: "stopped", Tags: "sablier", Node: "pve1"}, + {VMID: 102, Name: "unmanaged", Status: "running", Tags: "production", Node: "pve1"}, + }) + defer server.Close() + + p, err := proxmoxlxc.New(t.Context(), proxmoxlxc.NewTestClient(server.URL), slogt.New(t)) + assert.NilError(t, err) + + instances, err := p.InstanceList(t.Context(), provider.InstanceListOptions{All: false}) + assert.NilError(t, err) + + assert.DeepEqual(t, instances, []sablier.InstanceConfiguration{ + {Name: "web", Group: "frontend", Enabled: "true"}, + }) +} + +func TestProxmoxLXCProvider_InstanceList_MultiNode(t *testing.T) { + t.Parallel() + + server := proxmoxlxc.MockServer(t, []string{"pve1", "pve2"}, []proxmoxlxc.TestContainer{ + {VMID: 100, Name: "app1", Status: "running", Tags: "sablier;sablier-group-apps", Node: "pve1"}, + {VMID: 200, Name: "app2", Status: "stopped", Tags: "sablier;sablier-group-apps", Node: "pve2"}, + }) + defer server.Close() + + p, err := proxmoxlxc.New(t.Context(), proxmoxlxc.NewTestClient(server.URL), slogt.New(t)) + assert.NilError(t, err) + + instances, err := p.InstanceList(t.Context(), provider.InstanceListOptions{All: true}) + assert.NilError(t, err) + + sort.Slice(instances, func(i, j int) bool { return instances[i].Name < instances[j].Name }) + assert.DeepEqual(t, instances, []sablier.InstanceConfiguration{ + {Name: "app1", Group: "apps", Enabled: "true"}, + {Name: "app2", Group: "apps", Enabled: "true"}, + }) +} + +func TestProxmoxLXCProvider_InstanceList_DuplicateHostname(t *testing.T) { + t.Parallel() + + server := proxmoxlxc.MockServer(t, []string{"pve1", "pve2"}, []proxmoxlxc.TestContainer{ + {VMID: 100, Name: "web", Status: "running", Tags: "sablier", Node: "pve1"}, + {VMID: 200, Name: "web", Status: "stopped", Tags: "sablier", Node: "pve2"}, + }) + defer server.Close() + + p, err := proxmoxlxc.New(t.Context(), proxmoxlxc.NewTestClient(server.URL), slogt.New(t)) + assert.NilError(t, err) + + _, err = p.InstanceList(t.Context(), provider.InstanceListOptions{All: true}) + assert.ErrorContains(t, err, "duplicate hostname") +} + +func TestProxmoxLXCProvider_InstanceGroups(t *testing.T) { + t.Parallel() + + server := proxmoxlxc.MockServer(t, []string{"pve1"}, []proxmoxlxc.TestContainer{ + {VMID: 100, Name: "web1", Status: "running", Tags: "sablier;sablier-group-frontend", Node: "pve1"}, + {VMID: 101, Name: "web2", Status: "running", Tags: "sablier;sablier-group-frontend", Node: "pve1"}, + {VMID: 102, Name: "db", Status: "stopped", Tags: "sablier", Node: "pve1"}, + }) + defer server.Close() + + p, err := proxmoxlxc.New(t.Context(), proxmoxlxc.NewTestClient(server.URL), slogt.New(t)) + assert.NilError(t, err) + + groups, err := p.InstanceGroups(t.Context()) + assert.NilError(t, err) + + for _, v := range groups { + sort.Strings(v) + } + + assert.DeepEqual(t, groups, map[string][]string{ + "frontend": {"web1", "web2"}, + "default": {"db"}, + }) +} diff --git a/pkg/provider/proxmoxlxc/container_start.go b/pkg/provider/proxmoxlxc/container_start.go new file mode 100644 index 00000000..9cf79e07 --- /dev/null +++ b/pkg/provider/proxmoxlxc/container_start.go @@ -0,0 +1,45 @@ +package proxmoxlxc + +import ( + "context" + "fmt" + "log/slog" +) + +func (p *Provider) InstanceStart(ctx context.Context, name string) error { + ref, err := p.resolve(ctx, name) + if err != nil { + return fmt.Errorf("cannot resolve instance %q: %w", name, err) + } + + ct, err := p.getContainer(ctx, ref) + if err != nil { + return err + } + + if ct.Status == "running" { + p.l.DebugContext(ctx, "container already running", slog.String("name", ref.name), slog.Int("vmid", ref.vmid)) + return nil + } + + p.l.DebugContext(ctx, "starting container", slog.String("name", ref.name), slog.Int("vmid", ref.vmid), slog.String("node", ref.node)) + + task, err := ct.Start(ctx) + if err != nil { + return fmt.Errorf("cannot start container %q (VMID %d): %w", ref.name, ref.vmid, err) + } + + // Store the task so InstanceInspect can track its progress via Ping(). + // This mirrors the Docker provider pattern: InstanceStart returns quickly, + // and readiness is determined by polling InstanceInspect. + p.mu.Lock() + p.pendingTasks[name] = &pendingTask{task: task} + p.mu.Unlock() + + p.l.DebugContext(ctx, "start task submitted", + slog.String("name", ref.name), + slog.Int("vmid", ref.vmid), + slog.String("upid", string(task.UPID)), + ) + return nil +} diff --git a/pkg/provider/proxmoxlxc/container_start_test.go b/pkg/provider/proxmoxlxc/container_start_test.go new file mode 100644 index 00000000..dfece332 --- /dev/null +++ b/pkg/provider/proxmoxlxc/container_start_test.go @@ -0,0 +1,164 @@ +package proxmoxlxc_test + +import ( + "testing" + "time" + + "github.com/neilotoole/slogt" + "github.com/sablierapp/sablier/pkg/provider/proxmoxlxc" + "gotest.tools/v3/assert" +) + +func TestProxmoxLXCProvider_InstanceStart(t *testing.T) { + t.Parallel() + + server := proxmoxlxc.MockServer(t, []string{"pve1"}, []proxmoxlxc.TestContainer{ + {VMID: 100, Name: "web", Status: "stopped", Tags: "sablier", Node: "pve1"}, + }) + defer server.Close() + + p, err := proxmoxlxc.New(t.Context(), proxmoxlxc.NewTestClient(server.URL), slogt.New(t)) + assert.NilError(t, err) + + err = p.InstanceStart(t.Context(), "web") + assert.NilError(t, err) + + // Verify via InstanceInspect that the start was accepted. + got, err := p.InstanceInspect(t.Context(), "web") + assert.NilError(t, err) + assert.Equal(t, string(got.Status), "stopped") +} + +func TestProxmoxLXCProvider_InstanceStart_ByVMID(t *testing.T) { + t.Parallel() + + server := proxmoxlxc.MockServer(t, []string{"pve1"}, []proxmoxlxc.TestContainer{ + {VMID: 100, Name: "web", Status: "stopped", Tags: "sablier", Node: "pve1"}, + }) + defer server.Close() + + p, err := proxmoxlxc.New(t.Context(), proxmoxlxc.NewTestClient(server.URL), slogt.New(t)) + assert.NilError(t, err) + + err = p.InstanceStart(t.Context(), "100") + assert.NilError(t, err) +} + +func TestProxmoxLXCProvider_InstanceStart_NotFound(t *testing.T) { + t.Parallel() + + server := proxmoxlxc.MockServer(t, []string{"pve1"}, []proxmoxlxc.TestContainer{}) + defer server.Close() + + p, err := proxmoxlxc.New(t.Context(), proxmoxlxc.NewTestClient(server.URL), slogt.New(t)) + assert.NilError(t, err) + + err = p.InstanceStart(t.Context(), "nonexistent") + assert.ErrorContains(t, err, "not found") +} + +func TestProxmoxLXCProvider_InstanceStart_WithoutSablierTag(t *testing.T) { + t.Parallel() + + server := proxmoxlxc.MockServer(t, []string{"pve1"}, []proxmoxlxc.TestContainer{ + {VMID: 200, Name: "unmanaged", Status: "stopped", Tags: "", Node: "pve1"}, + }) + defer server.Close() + + p, err := proxmoxlxc.New(t.Context(), proxmoxlxc.NewTestClient(server.URL), slogt.New(t)) + assert.NilError(t, err) + + // Containers without the sablier tag can still be started by name, matching Docker behavior. + err = p.InstanceStart(t.Context(), "unmanaged") + assert.NilError(t, err) +} + +func TestProxmoxLXCProvider_InstanceStart_AlreadyRunning(t *testing.T) { + t.Parallel() + + server := proxmoxlxc.MockServer(t, []string{"pve1"}, []proxmoxlxc.TestContainer{ + {VMID: 100, Name: "web", Status: "running", Tags: "sablier", Node: "pve1"}, + }) + defer server.Close() + + p, err := proxmoxlxc.New(t.Context(), proxmoxlxc.NewTestClient(server.URL), slogt.New(t)) + assert.NilError(t, err) + + err = p.InstanceStart(t.Context(), "web") + assert.NilError(t, err) + + // Already running container should be ready immediately (no pending task). + got, err := p.InstanceInspect(t.Context(), "web") + assert.NilError(t, err) + assert.Equal(t, string(got.Status), "ready") +} + +func TestProxmoxLXCProvider_InstanceStart_TaskFailure(t *testing.T) { + t.Parallel() + + server := proxmoxlxc.MockServer(t, []string{"pve1"}, []proxmoxlxc.TestContainer{ + {VMID: 100, Name: "broken", Status: "stopped", Tags: "sablier", Node: "pve1", + StartTaskState: proxmoxlxc.TaskFailed, StartTaskExitStatus: "startup for container '100' failed"}, + }) + defer server.Close() + + p, err := proxmoxlxc.New(t.Context(), proxmoxlxc.NewTestClient(server.URL), slogt.New(t)) + assert.NilError(t, err) + + // InstanceStart should return nil (task is stored, not awaited). + err = p.InstanceStart(t.Context(), "broken") + assert.NilError(t, err) + + // InstanceInspect should detect the failed task via Ping() and report error state. + got, err := p.InstanceInspect(t.Context(), "broken") + assert.NilError(t, err) + assert.Equal(t, string(got.Status), "error") + assert.Assert(t, got.Message != "", "expected a non-empty error message") +} + +func TestProxmoxLXCProvider_InstanceStart_TaskFailureTTLExpiry(t *testing.T) { + t.Parallel() + + // Set the task end time far enough in the past that the TTL (30s) has expired. + server := proxmoxlxc.MockServer(t, []string{"pve1"}, []proxmoxlxc.TestContainer{ + {VMID: 100, Name: "broken", Status: "stopped", Tags: "sablier", Node: "pve1", + StartTaskState: proxmoxlxc.TaskFailed, + StartTaskExitStatus: "startup for container '100' failed", + StartTaskEndTime: time.Now().Add(-time.Minute)}, + }) + defer server.Close() + + p, err := proxmoxlxc.New(t.Context(), proxmoxlxc.NewTestClient(server.URL), slogt.New(t)) + assert.NilError(t, err) + + err = p.InstanceStart(t.Context(), "broken") + assert.NilError(t, err) + + // First InstanceInspect triggers Ping which discovers the failure. + // Since EndTime is >30s ago, the failed entry should be cleared and + // a fresh InstanceStart is attempted, leaving the instance in starting state. + got, err := p.InstanceInspect(t.Context(), "broken") + assert.NilError(t, err) + assert.Equal(t, string(got.Status), "starting", "expected starting after TTL expiry retry, got %s", got.Status) +} + +func TestProxmoxLXCProvider_InstanceStart_TaskInProgress(t *testing.T) { + t.Parallel() + + server := proxmoxlxc.MockServer(t, []string{"pve1"}, []proxmoxlxc.TestContainer{ + {VMID: 100, Name: "slow", Status: "stopped", Tags: "sablier", Node: "pve1", + StartTaskState: proxmoxlxc.TaskRunning}, + }) + defer server.Close() + + p, err := proxmoxlxc.New(t.Context(), proxmoxlxc.NewTestClient(server.URL), slogt.New(t)) + assert.NilError(t, err) + + err = p.InstanceStart(t.Context(), "slow") + assert.NilError(t, err) + + // InstanceInspect should report starting while the task is still running. + got, err := p.InstanceInspect(t.Context(), "slow") + assert.NilError(t, err) + assert.Equal(t, string(got.Status), "starting") +} diff --git a/pkg/provider/proxmoxlxc/container_stop.go b/pkg/provider/proxmoxlxc/container_stop.go new file mode 100644 index 00000000..4d96b927 --- /dev/null +++ b/pkg/provider/proxmoxlxc/container_stop.go @@ -0,0 +1,44 @@ +package proxmoxlxc + +import ( + "context" + "fmt" + "log/slog" + "time" +) + +func (p *Provider) InstanceStop(ctx context.Context, name string) error { + ref, err := p.resolve(ctx, name) + if err != nil { + return fmt.Errorf("cannot resolve instance %q: %w", name, err) + } + + ct, err := p.getContainer(ctx, ref) + if err != nil { + return err + } + + // Clear any pending start task — the stop supersedes it. + p.mu.Lock() + delete(p.pendingTasks, name) + p.mu.Unlock() + + if ct.Status == "stopped" { + p.l.DebugContext(ctx, "container already stopped", slog.String("name", ref.name), slog.Int("vmid", ref.vmid)) + return nil + } + + p.l.DebugContext(ctx, "stopping container", slog.String("name", ref.name), slog.Int("vmid", ref.vmid), slog.String("node", ref.node)) + + task, err := ct.Stop(ctx) + if err != nil { + return fmt.Errorf("cannot stop container %q (VMID %d): %w", ref.name, ref.vmid, err) + } + + if err := task.Wait(ctx, 1*time.Second, taskTimeout(ctx)); err != nil { + return fmt.Errorf("cannot wait for container %q (VMID %d) to stop: %w", ref.name, ref.vmid, err) + } + + p.l.DebugContext(ctx, "container stopped", slog.String("name", ref.name), slog.Int("vmid", ref.vmid)) + return nil +} diff --git a/pkg/provider/proxmoxlxc/container_stop_test.go b/pkg/provider/proxmoxlxc/container_stop_test.go new file mode 100644 index 00000000..abd4141f --- /dev/null +++ b/pkg/provider/proxmoxlxc/container_stop_test.go @@ -0,0 +1,37 @@ +package proxmoxlxc_test + +import ( + "testing" + + "github.com/neilotoole/slogt" + "github.com/sablierapp/sablier/pkg/provider/proxmoxlxc" + "gotest.tools/v3/assert" +) + +func TestProxmoxLXCProvider_InstanceStop(t *testing.T) { + t.Parallel() + + server := proxmoxlxc.MockServer(t, []string{"pve1"}, []proxmoxlxc.TestContainer{ + {VMID: 100, Name: "web", Status: "running", Tags: "sablier", Node: "pve1"}, + }) + defer server.Close() + + p, err := proxmoxlxc.New(t.Context(), proxmoxlxc.NewTestClient(server.URL), slogt.New(t)) + assert.NilError(t, err) + + err = p.InstanceStop(t.Context(), "web") + assert.NilError(t, err) +} + +func TestProxmoxLXCProvider_InstanceStop_NotFound(t *testing.T) { + t.Parallel() + + server := proxmoxlxc.MockServer(t, []string{"pve1"}, []proxmoxlxc.TestContainer{}) + defer server.Close() + + p, err := proxmoxlxc.New(t.Context(), proxmoxlxc.NewTestClient(server.URL), slogt.New(t)) + assert.NilError(t, err) + + err = p.InstanceStop(t.Context(), "nonexistent") + assert.ErrorContains(t, err, "not found") +} diff --git a/pkg/provider/proxmoxlxc/events.go b/pkg/provider/proxmoxlxc/events.go new file mode 100644 index 00000000..01604b09 --- /dev/null +++ b/pkg/provider/proxmoxlxc/events.go @@ -0,0 +1,109 @@ +package proxmoxlxc + +import ( + "context" + "log/slog" + "slices" + "time" + + "github.com/sablierapp/sablier/pkg/provider" + "github.com/sablierapp/sablier/pkg/sablier" +) + +// maxConsecutivePollErrors is the number of consecutive scan failures +// before InstanceEvents gives up and reports a terminal error. +const maxConsecutivePollErrors = 5 + +// InstanceEvents polls Proxmox for status changes and emits events when +// instances transition from running to stopped. Proxmox VE does not provide +// a real-time event stream, so polling is used. +func (p *Provider) InstanceEvents(ctx context.Context, opts provider.InstanceEventsOptions) sablier.InstanceEventStream { + eventsC := make(chan sablier.InstanceInfo) + errC := make(chan error, 1) + + wantStopped := len(opts.Types) == 0 || slices.Contains(opts.Types, provider.InstanceEventStopped) + + go func() { + defer close(eventsC) + defer close(errC) + + if !wantStopped { + <-ctx.Done() + return + } + + // Track previously seen running containers + running := make(map[string]bool) + + // Initial scan + discovered, err := p.scanContainers(ctx) + if err != nil { + p.l.ErrorContext(ctx, "initial container scan failed", slog.Any("error", err)) + } else { + for _, d := range discovered { + if d.status == "running" { + running[d.ref.name] = true + } + } + } + + ticker := time.NewTicker(p.pollInterval) + defer ticker.Stop() + + var consecutiveErrors int + + for { + select { + case <-ticker.C: + discovered, err := p.scanContainers(ctx) + if err != nil { + consecutiveErrors++ + p.l.WarnContext(ctx, "container scan failed during polling", + slog.Any("error", err), + slog.Int("consecutive_errors", consecutiveErrors), + ) + if consecutiveErrors >= maxConsecutivePollErrors { + p.l.ErrorContext(ctx, "too many consecutive poll errors, closing event stream", + slog.Int("max", maxConsecutivePollErrors), + ) + errC <- err + return + } + continue + } + + consecutiveErrors = 0 + + currentRunning := make(map[string]bool) + for _, d := range discovered { + if d.status == "running" { + currentRunning[d.ref.name] = true + } + } + + // Detect containers that were running but are no longer + for name := range running { + if !currentRunning[name] { + p.l.DebugContext(ctx, "container stopped detected", slog.String("name", name)) + select { + case eventsC <- sablier.InstanceInfo{ + Name: name, + CurrentReplicas: 0, + DesiredReplicas: p.desiredReplicas, + Status: sablier.InstanceStatusStopped, + }: + case <-ctx.Done(): + return + } + } + } + + running = currentRunning + case <-ctx.Done(): + return + } + } + }() + + return sablier.InstanceEventStream{Events: eventsC, Err: errC} +} diff --git a/pkg/provider/proxmoxlxc/events_test.go b/pkg/provider/proxmoxlxc/events_test.go new file mode 100644 index 00000000..63c6ecd7 --- /dev/null +++ b/pkg/provider/proxmoxlxc/events_test.go @@ -0,0 +1,49 @@ +package proxmoxlxc_test + +import ( + "context" + "testing" + "time" + + "github.com/neilotoole/slogt" + "github.com/sablierapp/sablier/pkg/provider" + "github.com/sablierapp/sablier/pkg/provider/proxmoxlxc" + "gotest.tools/v3/assert" +) + +func TestProxmoxLXCProvider_InstanceEvents_ContextCancel(t *testing.T) { + t.Parallel() + + server := proxmoxlxc.MockServer(t, []string{"pve1"}, []proxmoxlxc.TestContainer{ + {VMID: 100, Name: "web", Status: "running", Tags: "sablier", Node: "pve1"}, + }) + defer server.Close() + + p, err := proxmoxlxc.New(t.Context(), proxmoxlxc.NewTestClient(server.URL), slogt.New(t)) + assert.NilError(t, err) + + ctx, cancel := context.WithCancel(t.Context()) + stream := p.InstanceEvents(ctx, provider.InstanceEventsOptions{ + Types: []provider.InstanceEventType{provider.InstanceEventStopped}, + }) + + // Cancel context and verify both channels are closed + cancel() + + deadline := time.After(3 * time.Second) + eventsClosed, errClosed := false, false + for !eventsClosed || !errClosed { + select { + case _, ok := <-stream.Events: + if !ok { + eventsClosed = true + } + case _, ok := <-stream.Err: + if !ok { + errClosed = true + } + case <-deadline: + t.Fatal("timed out waiting for stream channels to close") + } + } +} diff --git a/pkg/provider/proxmoxlxc/export_test.go b/pkg/provider/proxmoxlxc/export_test.go new file mode 100644 index 00000000..66865298 --- /dev/null +++ b/pkg/provider/proxmoxlxc/export_test.go @@ -0,0 +1,48 @@ +package proxmoxlxc + +import ( + "context" + "log/slog" + "net/http/httptest" + "testing" + "time" + + proxmox "github.com/luthermonson/go-proxmox" +) + +// Exported aliases for external test packages. + +// TaskState controls how the mock API reports a task's status. +type TaskState = taskState + +const ( + TaskCompleted TaskState = taskCompleted + TaskRunning TaskState = taskRunning + TaskFailed TaskState = taskFailed +) + +// TestContainer represents a container in the mock API. +type TestContainer = testContainer + +// MockServer sets up a mock Proxmox API server with the given nodes and containers. +func MockServer(t *testing.T, nodes []string, containers []TestContainer) *httptest.Server { + return mockServer(t, nodes, containers) +} + +// NewTestClient creates a Proxmox client configured for the mock server. +func NewTestClient(serverURL string) *proxmox.Client { + return proxmox.NewClient( + serverURL+"/api2/json", + proxmox.WithAPIToken("test@pam!test", "test-secret"), + ) +} + +// NewForTest creates a Provider with a custom poll interval for testing. +func NewForTest(ctx context.Context, client *proxmox.Client, logger *slog.Logger, pollInterval time.Duration) (*Provider, error) { + p, err := New(ctx, client, logger) + if err != nil { + return nil, err + } + p.pollInterval = pollInterval + return p, nil +} diff --git a/pkg/provider/proxmoxlxc/integration_test.go b/pkg/provider/proxmoxlxc/integration_test.go new file mode 100644 index 00000000..5bb3ec5a --- /dev/null +++ b/pkg/provider/proxmoxlxc/integration_test.go @@ -0,0 +1,127 @@ +package proxmoxlxc_test + +import ( + "context" + "testing" + "time" + + "github.com/sablierapp/sablier/pkg/provider" + "github.com/sablierapp/sablier/pkg/sablier" + "gotest.tools/v3/assert" +) + +func TestProxmoxLXCProvider_Integration(t *testing.T) { + if testing.Short() { + t.Skip("skipping integration test in short mode") + } + + env := setupProxmox(t) + p := env.provider + ctx := t.Context() + + t.Run("InstanceList", func(t *testing.T) { + instances, err := p.InstanceList(ctx, provider.InstanceListOptions{All: true}) + assert.NilError(t, err) + + // The cloned container should appear in the list. + found := false + for _, inst := range instances { + if inst.Name == env.name { + found = true + assert.Equal(t, inst.Group, "test") // from "sablier-group-test" tag + break + } + } + assert.Assert(t, found, "expected container %q in instance list", env.name) + }) + + t.Run("InstanceGroups", func(t *testing.T) { + groups, err := p.InstanceGroups(ctx) + assert.NilError(t, err) + + names, ok := groups["test"] + assert.Assert(t, ok, "expected group 'test' to exist") + + found := false + for _, n := range names { + if n == env.name { + found = true + break + } + } + assert.Assert(t, found, "expected container %q in group 'test'", env.name) + }) + + t.Run("StartAndInspect", func(t *testing.T) { + // Container should be stopped initially (freshly cloned). + info, err := p.InstanceInspect(ctx, env.name) + assert.NilError(t, err) + assert.Equal(t, info.Status, sablier.InstanceStatusStopped) + + // Start the container. + err = p.InstanceStart(ctx, env.name) + assert.NilError(t, err) + + // Poll InstanceInspect until ready (task completion + IP assignment). + var ready bool + for i := 0; i < 30; i++ { + info, err = p.InstanceInspect(ctx, env.name) + assert.NilError(t, err) + + if info.Status == sablier.InstanceStatusReady { + ready = true + break + } + t.Logf("inspect attempt %d: status=%s", i+1, info.Status) + time.Sleep(2 * time.Second) + } + assert.Assert(t, ready, "expected container to become ready, last status: %s", info.Status) + assert.Equal(t, info.Name, env.name) + }) + + t.Run("Stop", func(t *testing.T) { + err := p.InstanceStop(ctx, env.name) + assert.NilError(t, err) + + info, err := p.InstanceInspect(ctx, env.name) + assert.NilError(t, err) + assert.Equal(t, info.Status, sablier.InstanceStatusStopped) + }) + + t.Run("InstanceEvents", func(t *testing.T) { + // Start the container first. + err := p.InstanceStart(ctx, env.name) + assert.NilError(t, err) + + // Wait until it's running. + for i := 0; i < 30; i++ { + info, err := p.InstanceInspect(ctx, env.name) + assert.NilError(t, err) + if info.Status == sablier.InstanceStatusReady { + break + } + time.Sleep(2 * time.Second) + } + + // Start the event stream with a cancelable context. + eventsCtx, cancel := context.WithCancel(ctx) + defer cancel() + + stream := p.InstanceEvents(eventsCtx, provider.InstanceEventsOptions{ + Types: []provider.InstanceEventType{provider.InstanceEventStopped}, + }) + + // Stop the container externally (simulate external stop). + err = p.InstanceStop(ctx, env.name) + assert.NilError(t, err) + + // Wait for the notification. + select { + case info, ok := <-stream.Events: + assert.Assert(t, ok, "events channel closed unexpectedly") + assert.Equal(t, info.Name, env.name) + case <-time.After(30 * time.Second): + t.Fatal("timed out waiting for stop notification") + } + }) +} diff --git a/pkg/provider/proxmoxlxc/proxmox_lxc.go b/pkg/provider/proxmoxlxc/proxmox_lxc.go new file mode 100644 index 00000000..44e2d43f --- /dev/null +++ b/pkg/provider/proxmoxlxc/proxmox_lxc.go @@ -0,0 +1,217 @@ +package proxmoxlxc + +import ( + "context" + "fmt" + "log/slog" + "strconv" + "strings" + "sync" + "time" + + proxmox "github.com/luthermonson/go-proxmox" + "github.com/sablierapp/sablier/pkg/sablier" +) + +// Interface guard +var _ sablier.Provider = (*Provider)(nil) + +// containerRef holds the mapping from an instance name to the actual Proxmox container location. +type containerRef struct { + node string + vmid int + name string // LXC hostname +} + +// failedTaskTTL is the duration after a task's EndTime (or first failure detection) +// before the failed entry is cleared from pendingTasks, allowing a fresh InstanceStart. +const failedTaskTTL = 30 * time.Second + +// pendingTask wraps a Proxmox task with the time failure was first detected. +// EndTime from the Proxmox API may be zero due to copier.Copy not preserving +// json:"-" fields, so failedAt serves as a reliable fallback. +type pendingTask struct { + task *proxmox.Task + failedAt time.Time // set once when IsFailed is first observed +} + +// Provider implements the sablier.Provider interface for Proxmox VE LXC containers. +type Provider struct { + client *proxmox.Client + l *slog.Logger + desiredReplicas int32 + pollInterval time.Duration + + mu sync.RWMutex + cache map[string]containerRef // hostname or VMID string → ref + pendingTasks map[string]*pendingTask // instance name → start task awaiting completion +} + +// New creates a new Proxmox LXC provider and verifies the connection. +func New(ctx context.Context, client *proxmox.Client, logger *slog.Logger) (*Provider, error) { + logger = logger.With(slog.String("provider", "proxmox_lxc")) + + version, err := client.Version(ctx) + if err != nil { + return nil, fmt.Errorf("cannot connect to Proxmox VE API: %w", err) + } + + logger.InfoContext(ctx, "connection established with Proxmox VE", + slog.String("version", version.Version), + slog.String("release", version.Release), + ) + + return &Provider{ + client: client, + l: logger, + desiredReplicas: 1, + pollInterval: 10 * time.Second, + cache: make(map[string]containerRef), + pendingTasks: make(map[string]*pendingTask), + }, nil +} + +// resolve looks up a container by hostname, VMID string, or "node/vmid" format. +// It first checks the cache, then rescans all nodes if not found. +func (p *Provider) resolve(ctx context.Context, name string) (containerRef, error) { + // Handle "node/vmid" format (e.g. "pve/111") + if node, vmidStr, ok := strings.Cut(name, "/"); ok { + vmid, err := strconv.Atoi(vmidStr) + if err != nil { + return containerRef{}, fmt.Errorf("invalid VMID in %q: %w", name, err) + } + // Try to resolve hostname from cache via VMID so that ref.name matches + // the hostname used by stop-event detection in InstanceEvents. + if ref, ok := p.lookupCache(vmidStr); ok && ref.node == node { + return ref, nil + } + // Cache miss for node/vmid — rescan containers to refresh hostname mapping. + if _, err := p.scanContainers(ctx); err != nil { + return containerRef{}, fmt.Errorf("cannot scan containers: %w", err) + } + if ref, ok := p.lookupCache(vmidStr); ok && ref.node == node { + return ref, nil + } + // Fall back to a best-effort reference when the container cannot be + // discovered via scan; name will be the original "node/vmid" string. + return containerRef{node: node, vmid: vmid, name: name}, nil + } + + if ref, ok := p.lookupCache(name); ok { + return ref, nil + } + + // Cache miss — rescan all nodes + if _, err := p.scanContainers(ctx); err != nil { + return containerRef{}, fmt.Errorf("cannot scan containers: %w", err) + } + + if ref, ok := p.lookupCache(name); ok { + return ref, nil + } + + return containerRef{}, fmt.Errorf("container %q not found", name) +} + +func (p *Provider) lookupCache(key string) (containerRef, bool) { + p.mu.RLock() + ref, ok := p.cache[key] + p.mu.RUnlock() + return ref, ok +} + +// getContainer fetches a proxmox.Container from the API for the given ref. +func (p *Provider) getContainer(ctx context.Context, ref containerRef) (*proxmox.Container, error) { + node, err := p.client.Node(ctx, ref.node) + if err != nil { + return nil, fmt.Errorf("cannot get node %q: %w", ref.node, err) + } + ct, err := node.Container(ctx, ref.vmid) + if err != nil { + return nil, fmt.Errorf("cannot get container %d on node %q: %w", ref.vmid, ref.node, err) + } + return ct, nil +} + +const defaultTaskTimeout = 60 * time.Second + +// taskTimeout returns the remaining time from the context deadline, or defaultTaskTimeout if none is set. +func taskTimeout(ctx context.Context) time.Duration { + if deadline, ok := ctx.Deadline(); ok { + if remaining := time.Until(deadline); remaining > 0 { + return remaining + } + } + return defaultTaskTimeout +} + +type discoveredContainer struct { + ref containerRef + tags []string + status string +} + +// scanContainers scans all nodes for LXC containers and rebuilds the cache. +// All containers are cached so that resolve() can find any container by name or VMID +// (matching Docker/Podman behavior where labels are not required for direct access). +// Returns only sablier-tagged containers for use by InstanceList/InstanceGroups. +func (p *Provider) scanContainers(ctx context.Context) ([]discoveredContainer, error) { + nodes, err := p.client.Nodes(ctx) + if err != nil { + return nil, fmt.Errorf("cannot list nodes: %w", err) + } + + var discovered []discoveredContainer + newCache := make(map[string]containerRef) + sablierNames := make(map[string]containerRef) // for duplicate detection among managed containers + + for _, ns := range nodes { + node, err := p.client.Node(ctx, ns.Node) + if err != nil { + p.l.WarnContext(ctx, "cannot access node, skipping", slog.String("node", ns.Node), slog.Any("error", err)) + continue + } + + containers, err := node.Containers(ctx) + if err != nil { + p.l.WarnContext(ctx, "cannot list containers on node, skipping", slog.String("node", ns.Node), slog.Any("error", err)) + continue + } + + for _, c := range containers { + ref := containerRef{ + node: ns.Node, + vmid: int(c.VMID), + name: c.Name, + } + + // Cache all containers for resolve() lookups. + newCache[c.Name] = ref + newCache[strconv.Itoa(int(c.VMID))] = ref + + tags := parseTags(c.Tags) + if !hasSablierTag(tags) { + continue + } + + // Check for hostname collision among sablier-managed containers. + if existing, ok := sablierNames[c.Name]; ok { + return nil, fmt.Errorf("duplicate hostname %q found on node %q (VMID %d) and node %q (VMID %d) among sablier-managed containers", + c.Name, existing.node, existing.vmid, ns.Node, int(c.VMID)) + } + sablierNames[c.Name] = ref + + discovered = append(discovered, discoveredContainer{ + ref: ref, + tags: tags, + status: c.Status, + }) + } + } + + p.mu.Lock() + p.cache = newCache + p.mu.Unlock() + + return discovered, nil +} diff --git a/pkg/provider/proxmoxlxc/tags.go b/pkg/provider/proxmoxlxc/tags.go new file mode 100644 index 00000000..2352dece --- /dev/null +++ b/pkg/provider/proxmoxlxc/tags.go @@ -0,0 +1,44 @@ +package proxmoxlxc + +import ( + "strings" + + proxmox "github.com/luthermonson/go-proxmox" +) + +const ( + enableTag = "sablier" + groupPrefix = "sablier-group-" +) + +// parseTags splits a semicolon-separated tag string into individual tags. +func parseTags(tagString string) []string { + if tagString == "" { + return nil + } + return strings.Split(tagString, proxmox.TagSeperator) +} + +// hasSablierTag returns true if the "sablier" tag is present in the list. +func hasSablierTag(tags []string) bool { + for _, t := range tags { + if t == enableTag { + return true + } + } + return false +} + +// extractGroup returns the group name from a "sablier-group-" tag. +// Returns "default" if no group tag is found. +func extractGroup(tags []string) string { + for _, t := range tags { + if strings.HasPrefix(t, groupPrefix) { + group := strings.TrimPrefix(t, groupPrefix) + if group != "" { + return group + } + } + } + return "default" +} diff --git a/pkg/provider/proxmoxlxc/tags_test.go b/pkg/provider/proxmoxlxc/tags_test.go new file mode 100644 index 00000000..96b5e082 --- /dev/null +++ b/pkg/provider/proxmoxlxc/tags_test.go @@ -0,0 +1,121 @@ +package proxmoxlxc + +import ( + "testing" + + "gotest.tools/v3/assert" +) + +func TestParseTags(t *testing.T) { + t.Parallel() + tests := []struct { + name string + input string + want []string + }{ + { + name: "empty string", + input: "", + want: nil, + }, + { + name: "single tag", + input: "sablier", + want: []string{"sablier"}, + }, + { + name: "multiple tags", + input: "sablier;sablier-group-web;production", + want: []string{"sablier", "sablier-group-web", "production"}, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + got := parseTags(tt.input) + assert.DeepEqual(t, got, tt.want) + }) + } +} + +func TestHasSablierTag(t *testing.T) { + t.Parallel() + tests := []struct { + name string + tags []string + want bool + }{ + { + name: "has sablier tag", + tags: []string{"sablier", "other"}, + want: true, + }, + { + name: "no sablier tag", + tags: []string{"other", "tags"}, + want: false, + }, + { + name: "empty tags", + tags: nil, + want: false, + }, + { + name: "sablier-group is not sablier", + tags: []string{"sablier-group-web"}, + want: false, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + got := hasSablierTag(tt.tags) + assert.Equal(t, got, tt.want) + }) + } +} + +func TestExtractGroup(t *testing.T) { + t.Parallel() + tests := []struct { + name string + tags []string + want string + }{ + { + name: "has group tag", + tags: []string{"sablier", "sablier-group-web"}, + want: "web", + }, + { + name: "no group tag defaults to default", + tags: []string{"sablier"}, + want: "default", + }, + { + name: "empty group prefix is ignored", + tags: []string{"sablier", "sablier-group-"}, + want: "default", + }, + { + name: "multiple group tags uses first", + tags: []string{"sablier-group-first", "sablier-group-second"}, + want: "first", + }, + { + name: "empty tags", + tags: nil, + want: "default", + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + got := extractGroup(tt.tags) + assert.Equal(t, got, tt.want) + }) + } +} diff --git a/pkg/provider/proxmoxlxc/testcontainers_test.go b/pkg/provider/proxmoxlxc/testcontainers_test.go new file mode 100644 index 00000000..300e97b3 --- /dev/null +++ b/pkg/provider/proxmoxlxc/testcontainers_test.go @@ -0,0 +1,173 @@ +package proxmoxlxc_test + +import ( + "context" + "crypto/tls" + "fmt" + "net/http" + "os" + "strconv" + "testing" + "time" + + proxmox "github.com/luthermonson/go-proxmox" + "github.com/neilotoole/slogt" + "github.com/sablierapp/sablier/pkg/provider/proxmoxlxc" +) + +// proxmoxTestEnv holds the state for an integration test against a real Proxmox VE. +type proxmoxTestEnv struct { + provider *proxmoxlxc.Provider + node string + vmid int // VMID of the cloned test container + name string // hostname of the cloned test container +} + +// setupProxmox creates a real Proxmox LXC container by cloning a template, +// tags it with "sablier" and "sablier-group-test", and returns a test environment. +// The cloned container is automatically destroyed via t.Cleanup. +// +// Required environment variables (test is skipped if any are missing): +// +// PROXMOX_URL - Proxmox API URL (e.g. https://proxmox.local:8006/api2/json) +// PROXMOX_TOKEN_ID - API token ID (e.g. root@pam!sablier) +// PROXMOX_TOKEN_SECRET - API token secret +// PROXMOX_TEST_NODE - Node name to run tests on +// PROXMOX_TEST_TEMPLATE_VMID - VMID of the LXC template to clone +// +// Optional: +// +// PROXMOX_TLS_INSECURE - Set to "true" to skip TLS verification +func setupProxmox(t *testing.T) *proxmoxTestEnv { + t.Helper() + + apiURL := os.Getenv("PROXMOX_URL") + tokenID := os.Getenv("PROXMOX_TOKEN_ID") + tokenSecret := os.Getenv("PROXMOX_TOKEN_SECRET") + nodeName := os.Getenv("PROXMOX_TEST_NODE") + templateVMIDStr := os.Getenv("PROXMOX_TEST_TEMPLATE_VMID") + + if apiURL == "" || tokenID == "" || tokenSecret == "" || nodeName == "" || templateVMIDStr == "" { + t.Skip("skipping integration test: PROXMOX_URL, PROXMOX_TOKEN_ID, PROXMOX_TOKEN_SECRET, PROXMOX_TEST_NODE, PROXMOX_TEST_TEMPLATE_VMID must be set") + } + + templateVMID, err := strconv.Atoi(templateVMIDStr) + if err != nil { + t.Fatalf("PROXMOX_TEST_TEMPLATE_VMID must be an integer: %v", err) + } + + ctx := t.Context() + + // Build client options + opts := []proxmox.Option{ + proxmox.WithAPIToken(tokenID, tokenSecret), + } + if os.Getenv("PROXMOX_TLS_INSECURE") == "true" { + opts = append(opts, proxmox.WithHTTPClient(&http.Client{ + Transport: &http.Transport{ + TLSClientConfig: &tls.Config{ + InsecureSkipVerify: true, //nolint:gosec // intentional for test environments + }, + }, + })) + } + + client := proxmox.NewClient(apiURL, opts...) + + // Verify connectivity + version, err := client.Version(ctx) + if err != nil { + t.Fatalf("cannot connect to Proxmox VE at %s: %v", apiURL, err) + } + t.Logf("connected to Proxmox VE %s (release %s)", version.Version, version.Release) + + // Get the node and template container + node, err := client.Node(ctx, nodeName) + if err != nil { + t.Fatalf("cannot get node %q: %v", nodeName, err) + } + + template, err := node.Container(ctx, templateVMID) + if err != nil { + t.Fatalf("cannot get template container %d: %v", templateVMID, err) + } + + // Clone the template + hostname := fmt.Sprintf("sablier-test-%d", time.Now().UnixMilli()%100000) + newID, task, err := template.Clone(ctx, &proxmox.ContainerCloneOptions{ + Hostname: hostname, + }) + if err != nil { + t.Fatalf("cannot clone template %d: %v", templateVMID, err) + } + + t.Logf("cloning template %d → VMID %d (hostname %s)", templateVMID, newID, hostname) + if err := task.Wait(ctx, 2*time.Second, 120*time.Second); err != nil { + t.Fatalf("clone task failed: %v", err) + } + + // Get the cloned container + ct, err := node.Container(ctx, newID) + if err != nil { + t.Fatalf("cannot get cloned container %d: %v", newID, err) + } + + // Add sablier tags + for _, tag := range []string{"sablier", "sablier-group-test"} { + tagTask, err := ct.AddTag(ctx, tag) + if err != nil { + t.Fatalf("cannot add tag %q to container %d: %v", tag, newID, err) + } + if tagTask != nil { + if err := tagTask.Wait(ctx, 1*time.Second, 30*time.Second); err != nil { + t.Fatalf("add tag %q task failed: %v", tag, err) + } + } + } + + t.Logf("test container ready: VMID %d, hostname %s, tags: sablier;sablier-group-test", newID, hostname) + + // Register cleanup: stop (if running) and destroy the cloned container. + // Use a background context because t.Context() is canceled when the test finishes. + t.Cleanup(func() { + cleanupCtx := context.Background() + // Re-fetch the container to get current status + ct, err := node.Container(cleanupCtx, newID) + if err != nil { + t.Logf("cleanup: cannot get container %d (may already be deleted): %v", newID, err) + return + } + + if ct.Status == "running" { + stopTask, err := ct.Stop(cleanupCtx) + if err != nil { + t.Logf("cleanup: cannot stop container %d: %v", newID, err) + } else if err := stopTask.Wait(cleanupCtx, 2*time.Second, 60*time.Second); err != nil { + t.Logf("cleanup: stop task failed for container %d: %v", newID, err) + } + } + + delTask, err := ct.Delete(cleanupCtx) + if err != nil { + t.Logf("cleanup: cannot delete container %d: %v", newID, err) + return + } + if err := delTask.Wait(cleanupCtx, 2*time.Second, 60*time.Second); err != nil { + t.Logf("cleanup: delete task failed for container %d: %v", newID, err) + } + t.Logf("cleanup: container %d deleted", newID) + }) + + // Create the provider with a shorter poll interval for integration tests. + p, err := proxmoxlxc.NewForTest(ctx, client, slogt.New(t), 2*time.Second) + if err != nil { + t.Fatalf("cannot create provider: %v", err) + } + + return &proxmoxTestEnv{ + provider: p, + node: nodeName, + vmid: newID, + name: hostname, + } +} diff --git a/pkg/provider/proxmoxlxc/testhelper_test.go b/pkg/provider/proxmoxlxc/testhelper_test.go new file mode 100644 index 00000000..fbbaf841 --- /dev/null +++ b/pkg/provider/proxmoxlxc/testhelper_test.go @@ -0,0 +1,206 @@ +package proxmoxlxc + +import ( + "encoding/json" + "fmt" + "net/http" + "net/http/httptest" + "strings" + "testing" + "time" +) + +// taskState controls how the mock API reports a task's status. +type taskState int + +const ( + taskCompleted taskState = iota // "stopped" with "OK" (default) + taskRunning // "running" (still in progress) + taskFailed // "stopped" with custom exit status +) + +// proxmoxResponse wraps data in the Proxmox API JSON envelope. +func proxmoxResponse(data interface{}) []byte { + b, _ := json.Marshal(map[string]interface{}{"data": data}) + return b +} + +// writeJSON writes a Proxmox API JSON response to w, failing the test on error. +func writeJSON(t *testing.T, w http.ResponseWriter, data interface{}) { + t.Helper() + w.Header().Set("Content-Type", "application/json") + if _, err := w.Write(proxmoxResponse(data)); err != nil { + t.Errorf("failed to write mock response: %v", err) + } +} + +// testContainer represents a container in the mock API. +type testContainer struct { + VMID int `json:"vmid"` + Name string `json:"name"` + Status string `json:"status"` + Tags string `json:"tags"` + Node string `json:"-"` // not sent in API response, used for routing + StartTaskState taskState `json:"-"` // controls mock task status response + StartTaskExitStatus string `json:"-"` // exit status when StartTaskState is taskFailed + StartTaskEndTime time.Time `json:"-"` // if set, used as task endtime; otherwise uses time of first completion response +} + +// mockServer sets up a mock Proxmox API server with the given nodes and containers. +func mockServer(t *testing.T, nodes []string, containers []testContainer) *httptest.Server { + t.Helper() + + // Build per-node container lists + nodeContainers := make(map[string][]testContainer) + for _, c := range containers { + nodeContainers[c.Node] = append(nodeContainers[c.Node], c) + } + + mux := http.NewServeMux() + + // GET /api2/json/version + mux.HandleFunc("GET /api2/json/version", func(w http.ResponseWriter, r *http.Request) { + writeJSON(t, w, map[string]string{ + "version": "8.2-1", + "release": "8.2", + "repoid": "test", + }) + }) + + // GET /api2/json/nodes + mux.HandleFunc("GET /api2/json/nodes", func(w http.ResponseWriter, r *http.Request) { + var ns []map[string]interface{} + for _, n := range nodes { + ns = append(ns, map[string]interface{}{ + "node": n, + "status": "online", + "type": "node", + }) + } + writeJSON(t, w, ns) + }) + + // Per-node container list and operations + for _, nodeName := range nodes { + node := nodeName // capture + + // GET /api2/json/nodes/{node}/status (called by client.Node()) + mux.HandleFunc(fmt.Sprintf("GET /api2/json/nodes/%s/status", node), func(w http.ResponseWriter, r *http.Request) { + writeJSON(t, w, map[string]interface{}{"node": node, "status": "online"}) + }) + + // GET /api2/json/nodes/{node}/lxc + mux.HandleFunc(fmt.Sprintf("GET /api2/json/nodes/%s/lxc", node), func(w http.ResponseWriter, r *http.Request) { + writeJSON(t, w, nodeContainers[node]) + }) + + // GET /api2/json/nodes/{node}/lxc/{vmid}/status/current and /config + for _, ct := range nodeContainers[node] { + c := ct // capture + mux.HandleFunc(fmt.Sprintf("GET /api2/json/nodes/%s/lxc/%d/status/current", node, c.VMID), func(w http.ResponseWriter, r *http.Request) { + writeJSON(t, w, map[string]interface{}{ + "vmid": c.VMID, + "name": c.Name, + "status": c.Status, + "tags": c.Tags, + "node": node, + }) + }) + + // GET /api2/json/nodes/{node}/lxc/{vmid}/config (called by node.Container()) + mux.HandleFunc(fmt.Sprintf("GET /api2/json/nodes/%s/lxc/%d/config", node, c.VMID), func(w http.ResponseWriter, r *http.Request) { + writeJSON(t, w, map[string]interface{}{ + "hostname": c.Name, + "tags": c.Tags, + }) + }) + + // GET /api2/json/nodes/{node}/lxc/{vmid}/interfaces + mux.HandleFunc(fmt.Sprintf("GET /api2/json/nodes/%s/lxc/%d/interfaces", node, c.VMID), func(w http.ResponseWriter, r *http.Request) { + if c.Status == "running" { + writeJSON(t, w, []map[string]string{ + {"name": "lo", "hwaddr": "00:00:00:00:00:00", "inet": "127.0.0.1/8", "inet6": "::1/128"}, + {"name": "eth0", "hwaddr": "bc:24:11:89:67:07", "inet": "192.168.1.100/24", "inet6": "fe80::1/64"}, + }) + } else { + // Stopped containers have no interfaces + writeJSON(t, w, []map[string]string{}) + } + }) + + // POST /api2/json/nodes/{node}/lxc/{vmid}/status/start + mux.HandleFunc(fmt.Sprintf("POST /api2/json/nodes/%s/lxc/%d/status/start", node, c.VMID), func(w http.ResponseWriter, r *http.Request) { + upid := fmt.Sprintf("UPID:%s:%08X:%08X:%08X:vzstart:%d:root@pam:", node, 1, 1, 1, c.VMID) + writeJSON(t, w, upid) + }) + + // POST /api2/json/nodes/{node}/lxc/{vmid}/status/stop + mux.HandleFunc(fmt.Sprintf("POST /api2/json/nodes/%s/lxc/%d/status/stop", node, c.VMID), func(w http.ResponseWriter, r *http.Request) { + upid := fmt.Sprintf("UPID:%s:%08X:%08X:%08X:vzstop:%d:root@pam:", node, 1, 1, 1, c.VMID) + writeJSON(t, w, upid) + }) + } + + // Build a VMID → task config map for this node. + type taskConfig struct { + state taskState + exitStatus string + endTime time.Time + } + taskConfigs := make(map[string]taskConfig) // vmid string → config + for _, c := range nodeContainers[node] { + if c.StartTaskState != taskCompleted { + taskConfigs[fmt.Sprintf("%d", c.VMID)] = taskConfig{ + state: c.StartTaskState, + exitStatus: c.StartTaskExitStatus, + endTime: c.StartTaskEndTime, + } + } + } + + // GET /api2/json/nodes/{node}/tasks/{upid}/status - task status + // The Task.UnmarshalJSON uses copier.Copy which zeroes fields not in the response, + // so we must include upid/node/type/id/user to preserve them (matching real Proxmox API). + tasksPrefix := fmt.Sprintf("/api2/json/nodes/%s/tasks/", node) + mux.HandleFunc(fmt.Sprintf("GET %s", tasksPrefix), func(w http.ResponseWriter, r *http.Request) { + // Extract UPID from the URL path: /api2/json/nodes/{node}/tasks/{upid}/status + rest := strings.TrimPrefix(r.URL.Path, tasksPrefix) + upid := strings.TrimSuffix(rest, "/status") + + // Determine task status from UPID (format: UPID:node:pid:pstart:time:type:vmid:user:) + status := "stopped" + exitStatus := "OK" + endTime := time.Now() + parts := strings.Split(upid, ":") + if len(parts) >= 7 { + if tc, ok := taskConfigs[parts[6]]; ok { + switch tc.state { + case taskRunning: + status = "running" + exitStatus = "" + case taskFailed: + exitStatus = tc.exitStatus + } + if !tc.endTime.IsZero() { + endTime = tc.endTime + } + } + } + + resp := map[string]interface{}{ + "status": status, + "exitstatus": exitStatus, + "upid": upid, + "node": node, + "type": "lxcstart", + "id": "100", + "user": "root@pam", + "starttime": endTime.Add(-2 * time.Second).Unix(), + "endtime": endTime.Unix(), + } + writeJSON(t, w, resp) + }) + } + + return httptest.NewServer(mux) +} diff --git a/pkg/sabliercmd/provider.go b/pkg/sabliercmd/provider.go index f339a522..de27e502 100644 --- a/pkg/sabliercmd/provider.go +++ b/pkg/sabliercmd/provider.go @@ -2,15 +2,19 @@ package sabliercmd import ( "context" + "crypto/tls" "fmt" "log/slog" + "net/http" + proxmox "github.com/luthermonson/go-proxmox" "github.com/moby/moby/client" "github.com/sablierapp/sablier/pkg/config" "github.com/sablierapp/sablier/pkg/provider/docker" "github.com/sablierapp/sablier/pkg/provider/dockerswarm" "github.com/sablierapp/sablier/pkg/provider/kubernetes" "github.com/sablierapp/sablier/pkg/provider/podman" + "github.com/sablierapp/sablier/pkg/provider/proxmoxlxc" "github.com/sablierapp/sablier/pkg/sablier" k8s "k8s.io/client-go/kubernetes" "k8s.io/client-go/rest" @@ -57,6 +61,21 @@ func setupProvider(ctx context.Context, logger *slog.Logger, config config.Provi return nil, fmt.Errorf("cannot create podman client: %w", err) } return podman.New(ctx, cli, logger) + case "proxmox_lxc": + opts := []proxmox.Option{ + proxmox.WithAPIToken(config.ProxmoxLXC.TokenID, config.ProxmoxLXC.TokenSecret), + } + if config.ProxmoxLXC.TLSInsecure { + transport := http.DefaultTransport.(*http.Transport).Clone() + transport.TLSClientConfig = &tls.Config{ + InsecureSkipVerify: true, //nolint:gosec // user-configured option for self-signed certs + } + opts = append(opts, proxmox.WithHTTPClient(&http.Client{ + Transport: transport, + })) + } + cli := proxmox.NewClient(config.ProxmoxLXC.URL, opts...) + return proxmoxlxc.New(ctx, cli, logger) } return nil, fmt.Errorf("unimplemented provider %s", config.Name) } diff --git a/pkg/sabliercmd/root.go b/pkg/sabliercmd/root.go index e2307c53..2dcfe1b0 100644 --- a/pkg/sabliercmd/root.go +++ b/pkg/sabliercmd/root.go @@ -52,6 +52,14 @@ It provides integrations with multiple reverse proxies and different loading str _ = viper.BindPFlag("provider.podman.uri", startCmd.Flags().Lookup("provider.podman.uri")) startCmd.Flags().StringVar(&conf.Provider.Docker.Strategy, "provider.docker.strategy", "stop", "Strategy to use to stop docker containers (stop or pause)") _ = viper.BindPFlag("provider.docker.strategy", startCmd.Flags().Lookup("provider.docker.strategy")) + startCmd.Flags().StringVar(&conf.Provider.ProxmoxLXC.URL, "provider.proxmox-lxc.url", "", "Proxmox VE API URL (e.g. https://proxmox:8006/api2/json)") + _ = viper.BindPFlag("provider.proxmox-lxc.url", startCmd.Flags().Lookup("provider.proxmox-lxc.url")) + startCmd.Flags().StringVar(&conf.Provider.ProxmoxLXC.TokenID, "provider.proxmox-lxc.token-id", "", "Proxmox VE API token ID (e.g. root@pam!sablier)") + _ = viper.BindPFlag("provider.proxmox-lxc.token-id", startCmd.Flags().Lookup("provider.proxmox-lxc.token-id")) + startCmd.Flags().StringVar(&conf.Provider.ProxmoxLXC.TokenSecret, "provider.proxmox-lxc.token-secret", "", "Proxmox VE API token secret") + _ = viper.BindPFlag("provider.proxmox-lxc.token-secret", startCmd.Flags().Lookup("provider.proxmox-lxc.token-secret")) + startCmd.Flags().BoolVar(&conf.Provider.ProxmoxLXC.TLSInsecure, "provider.proxmox-lxc.tls-insecure", false, "Skip TLS certificate verification for Proxmox VE API") + _ = viper.BindPFlag("provider.proxmox-lxc.tls-insecure", startCmd.Flags().Lookup("provider.proxmox-lxc.tls-insecure")) // Server flags startCmd.Flags().IntVar(&conf.Server.Port, "server.port", 10000, "The server port to use") diff --git a/pkg/sabliercmd/testdata/config_cli_wanted.json b/pkg/sabliercmd/testdata/config_cli_wanted.json index b71e5c5c..ecd4ee81 100644 --- a/pkg/sabliercmd/testdata/config_cli_wanted.json +++ b/pkg/sabliercmd/testdata/config_cli_wanted.json @@ -19,6 +19,12 @@ }, "Docker": { "Strategy": "pause" + }, + "ProxmoxLXC": { + "URL": "", + "TokenID": "", + "TokenSecret": "", + "TLSInsecure": false } }, "Sessions": { diff --git a/pkg/sabliercmd/testdata/config_default.json b/pkg/sabliercmd/testdata/config_default.json index afdb9ead..cc8098c5 100644 --- a/pkg/sabliercmd/testdata/config_default.json +++ b/pkg/sabliercmd/testdata/config_default.json @@ -19,6 +19,12 @@ }, "Docker": { "Strategy": "stop" + }, + "ProxmoxLXC": { + "URL": "", + "TokenID": "", + "TokenSecret": "", + "TLSInsecure": false } }, "Sessions": { diff --git a/pkg/sabliercmd/testdata/config_env_wanted.json b/pkg/sabliercmd/testdata/config_env_wanted.json index 8da50a78..c70b22a9 100644 --- a/pkg/sabliercmd/testdata/config_env_wanted.json +++ b/pkg/sabliercmd/testdata/config_env_wanted.json @@ -19,6 +19,12 @@ }, "Docker": { "Strategy": "pause" + }, + "ProxmoxLXC": { + "URL": "", + "TokenID": "", + "TokenSecret": "", + "TLSInsecure": false } }, "Sessions": { diff --git a/pkg/sabliercmd/testdata/config_yaml_wanted.json b/pkg/sabliercmd/testdata/config_yaml_wanted.json index 1494211c..e318fd96 100644 --- a/pkg/sabliercmd/testdata/config_yaml_wanted.json +++ b/pkg/sabliercmd/testdata/config_yaml_wanted.json @@ -19,6 +19,12 @@ }, "Docker": { "Strategy": "pause" + }, + "ProxmoxLXC": { + "URL": "", + "TokenID": "", + "TokenSecret": "", + "TLSInsecure": false } }, "Sessions": {