Skip to content

OpenStack Hibernation #2722

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
msu8 wants to merge 11 commits into
base: master
Choose a base branch
from
Open

OpenStack Hibernation #2722

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
4e05854
Implement OpenStack hibernation support
msu8 Jul 12, 2025
d177558
Add tests, better error handling, resource checking, finalize scripts
msu8 Jul 28, 2025
07cf30b
add docs, more changes
msu8 Jul 28, 2025
f3004f0
Merge branch 'openshift:master' into openstack_hibernation
msu8 Jul 28, 2025
cd33a48
- Address first wave of review feedback
msu8 Aug 11, 2025
546b625
Merge branch 'openstack_hibernation' of github.com:msu8/hive into ope…
msu8 Aug 11, 2025
541b6dd
Remove quota validation, structs, client creation from upstream, snap…
msu8 Sep 12, 2025
f2ca9e3
Merge branch 'master' into openstack_hibernation
msu8 Sep 24, 2025
33efb34
fix missing func for testing, try to fix dependencies
msu8 Oct 2, 2025
0728771
address idempotency, aggregate errors, filter using tags, handle 404/…
msu8 Nov 4, 2025
4e7ce15
run update-gofmt
msu8 Nov 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 20 additions & 1 deletion docs/hibernating-clusters.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
- [Handling Incompatible Cloud Provider](#handling-incompatible-cloud-provider)
- [Approving CSRs](#approving-csrs)
- [Resuming from a Hibernating State](#resuming-from-a-hibernating-state)
- [OpenStack Hibernation](#openstack-hibernation)

# Hibernating Clusters

Expand Down Expand Up @@ -345,4 +346,22 @@ The unreachable controller tracks the reachability of a cluster and is responsib

Once a cluster hibernates and stops responding, the unreachable controller sets the `Unreachable` condition's `Status` to `True`.

Once the cluster resumes, the unreachable controller set the condition `False` and the hive controllers are free to resume syncing.
Once the cluster resumes, the unreachable controller set the condition `False` and the hive controllers are free to resume syncing.
Copy link
Member

@2uasimojo 2uasimojo Oct 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Latent)

Suggested change
Once the cluster resumes, the unreachable controller set the condition `False` and the hive controllers are free to resume syncing.
Once the cluster resumes, the unreachable controller sets the condition `False` and the hive controllers are free to resume syncing.


## OpenStack Hibernation
OpenStack hibernation deletes cluster instances while preserving the ability to restore them. Unlike other cloud providers, OpenStack lacks native hibernation, therefore an implementation based on snapshots is necessary.

While OpenStack provides shelving functionality, replicating true hibernation behavior similar to other cloud providers is difficult and often not possible depending on the OpenStack deployment configuration. Hive's snapshot-based approach provides a universally compatible solution that works with any OpenStack setup.

OpenStack hibernation creates snapshots of all instances, saves their complete configuration into a secret (CD namespace), and deletes the instances, completely freeing all project resources consumed by the cluster.

Restoration recreates instances with identical attributes to their original state.

Due to the specific nature of the snapshot based implementation, the hibernation and restoration process inherently requires more time to finish compared to other cloud providers.

Once the ClusterPool is created and the cluster is claimed, the workflow is similar to other implementations:

```bash
$ oc patch cd $cd -n $cd_ns --type='merge' -p $'spec:\n powerState: Hibernating'
$ oc patch cd $cd -n $cd_ns --type='merge' -p $'spec:\n powerState: Running'
```
Copy link
Member

@2uasimojo 2uasimojo Jul 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this chunk isn't needed.

  • Hibernation is used automatically for unclaimed clusters in ClusterPools whenever size > runningCount.
  • Hibernation isn't specific to ClusterPools -- it can be used for standalone ClusterDeployments.
  • These patch commands are already listed above (~L35).

1 change: 1 addition & 0 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,7 @@ require (
)

require (
github.com/gophercloud/gophercloud v1.7.0
github.com/gophercloud/utils/v2 v2.0.0-20250212084022-725b94822eeb
gopkg.in/evanphx/json-patch.v4 v4.12.0
)
Expand Down
3 changes: 3 additions & 0 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -510,6 +510,8 @@ github.com/googleapis/enterprise-certificate-proxy v0.3.4 h1:XYIDZApgAnrN1c855gT
github.com/googleapis/enterprise-certificate-proxy v0.3.4/go.mod h1:YKe7cfqYXjKGpGvmSg28/fFvhNzinZQm8DGnaburhGA=
github.com/googleapis/gax-go/v2 v2.14.1 h1:hb0FFeiPaQskmvakKu5EbCbpntQn48jyHuvrkurSS/Q=
github.com/googleapis/gax-go/v2 v2.14.1/go.mod h1:Hb/NubMaVM88SrNkvl8X/o8XWwDJEPqouaLeN2IUxoA=
github.com/gophercloud/gophercloud v1.7.0 h1:fyJGKh0LBvIZKLvBWvQdIgkaV5yTM3Jh9EYUh+UNCAs=
github.com/gophercloud/gophercloud v1.7.0/go.mod h1:aAVqcocTSXh2vYFZ1JTvx4EQmfgzxRcNupUfxZbBNDM=
github.com/gophercloud/gophercloud/v2 v2.5.0 h1:DubPfC43gsZiGZ9LT1IJflVMm+0rck0ejoPsH8D5rqk=
github.com/gophercloud/gophercloud/v2 v2.5.0/go.mod h1:Ki/ILhYZr/5EPebrPL9Ej+tUg4lqx71/YH2JWVeU+Qk=
github.com/gophercloud/utils/v2 v2.0.0-20250212084022-725b94822eeb h1:TQTXVYXL3d0zRAybRUKKboO0z/XAsXEfU6Oax8n00kc=
Expand Down Expand Up @@ -1077,6 +1079,7 @@ golang.org/x/crypto v0.0.0-20190530122614-20be4c3c3ed5/go.mod h1:yigFU9vqHzYiE8U
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.0.0-20220722155217-630584e8d5aa/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.0.0-20220829220503-c86fa9a7ed90/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.6.0/go.mod h1:OFC/31mSvZgRz0V1QTNCzfAI1aIRzbiufJtkMIlEp58=
golang.org/x/crypto v0.17.0/go.mod h1:gCAAfMLgwOJRpTjQ2zCCt2OcSfYMTeZVSRtQlPC7Nq4=
golang.org/x/crypto v0.19.0/go.mod h1:Iy9bg/ha4yyC70EfRS8jz+B6ybOBKMaSxLj6P6oBDfU=
Expand Down
Loading