✨ ROSANetwork: new CRD & reconciler to provision network infrastructure for ROSA-HCP #5464

mzazrivec · 2025-04-09T19:27:34Z

What type of PR is this?

/kind feature

What this PR does / why we need it:

This pull request implements CRD and a controller for provisioning complete networking infrastructure required to install a ROSA-HCP cluster in AWS. The proposal for this implementation has been described in #5381.

Under the hood, the implementation uses cloudformation stack and a static (i.e. no possibility of customization) cloudformation template from rosa-cli

This pull request depends on openshift/rosa#2904 (now merged).

Quick howto:

$ export ROSA_NETWORK_NAME=rosa-net-01
$ export AWS_REGION=us-west-2
$ export AVAILABILITY_ZONE_COUNT=2
$ export CIDR_BLOCK=10.0.0.0/16
$ clusterctl generate yaml --from templates/rosa-network.yaml > rosa-net-01.yaml
$ kubectl apply -f rosa-net-01.yaml

To use the ROSANetwork from ROSA control plane:

apiVersion: controlplane.cluster.x-k8s.io/v1beta2
kind: ROSAControlPlane
metadata:
  name: rosa-hcp01-control-plane
  namespace: default
spec:
  rosaNetworkRef:
    name: rosa-net01

and skip / remove subnets and availability zones from the CP spec.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Special notes for your reviewer:

Checklist:

Release note:

New API for provisioning network infrastructure for ROSA clusters

k8s-ci-robot · 2025-04-09T19:27:44Z

Hi @mzazrivec. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

serngawy · 2025-04-10T14:39:00Z

config/crd/patches/webhook_in_rosanetworks.yaml

+    webhookClientConfig:
+      # this is "\n" used as a placeholder, otherwise it will be rejected by the apiserver for being blank,
+      # but we're going to set it later using the cert-manager (or potentially a patch if not using cert-manager)
+      caBundle: Cg==


No need to add the caBundle.

serngawy · 2025-04-10T14:45:18Z

exp/api/v1beta2/rosanetwork_types.go

+	Resource string `json:"resource"`
+
+	// Identified of the created resource. Will be filled in once the resource is created & ready
+	ID string `json:"ID"`


Suggested change

ID string `json:"ID"`

Id string `json:"id"`

Or resourceId

serngawy · 2025-04-10T14:46:01Z

exp/api/v1beta2/rosanetwork_types.go

+// CFResource groups information pertaining to a resource created as a part of a cloudformation stack
+type CFResource struct {
+	// Name of the created resource: NATGateway1, VPC, SecurityGroup, ...
+	Resource string `json:"resource"`


Suggested change

Resource string `json:"resource"`

Name string `json:"name"`

OR resourceName

serngawy · 2025-04-10T14:49:19Z

exp/api/v1beta2/rosanetwork_types.go

+	Status string `json:"status"`
+
+	// Message pertaining to the status of the resource
+	Reason string `json:"reason"`


message is better I guess ?

Suggested change

Reason string `json:"reason"`

Message string `json:"message"`

serngawy · 2025-04-10T14:51:37Z

exp/api/v1beta2/rosanetwork_types.go

+	// Availability zone of the subnet pair
+	AvailabilityZone string `json:"availabilityZone"`
+
+	// ID of the public subnet


Suggested change

// ID of the public subnet

// Public subnet Id ex; subnet-xxxxxxxxxx

serngawy · 2025-04-10T14:53:59Z

main.go

 		}
 	}

+	// TODO: feature gates?


I don't think we need a new feature gate, we can have it under ROSA feature gate

Yes. I did not mean a new feature gate here, just the existing rosa FG.

serngawy · 2025-04-23T11:46:57Z

you also need to update the ValidatingWebhookConfiguration and MutatingWebhookConfiguration here

serngawy

/ok-to-test

serngawy · 2025-04-24T13:43:59Z

exp/api/v1beta2/rosanetwork_types.go

+	// If no identity is specified, the default identity for this controller will be used.
+	//
+	// +optional
+	IdentityRef *infrav1.AWSIdentityReference `json:"identityRef,omitempty"`


Not sure, if we want to provide this option to end user. We don't do that with RosaControlPlane only default aws identity. However, we should provide OCM identityRef

Why shouldn't we provide this option to the end user? We need to specify the ref to the aws secret somehow. Here I'm just reusing existing structures & code.

What do you mean by OCM identity ref? OCM will not be involved here in any way.

well, to use openshift/rosa and establish ocm client you need to have ocm authentication. Is this not the case with the RosaNetwork CF stack creation?

No. No OCM credentials are needed for rosanet, just AWS credentials.

@serngawy Are you satisfied with the answers here?

@mzazrivec I do remember we discuss that, but after checking the ROSANetwork cloud formation stack template , there are tags added as rosa_hcp_policy and roas service here.
Those tags I think is used to check for privileges ?
I think we have to authenticate the ocm credential. Even if we don't need to create the CF stack but enduser must be a valid OCM user.

@serngawy What does creating VPC with certain tags and checking OCM credentials have to do with each other?

okay, as we discussed no need to have ocm authentication.

controlplane/rosa/api/v1beta2/rosacontrolplane_types.go

damdo · 2025-09-30T10:11:19Z

pkg/cloud/scope/session.go

 func getSessionName(region string, clusterScoper cloud.SessionMetadata) string {
-	return fmt.Sprintf("%s-%s-%s", region, clusterScoper.InfraClusterName(), clusterScoper.Namespace())
+	return fmt.Sprintf("%s-%s-%s-%s", region, clusterScoper.ControllerName(), clusterScoper.InfraClusterName(), clusterScoper.Namespace())
 }


This changes the session name in all the places already using this (not only ROSA ones). Is this a backward compatible change? (cc. @richardcase @punkwalker)

I guess it should be okay as when the capa-manager update happen all cache sessions will be re-created with the new session name.
Does the cache stored some how even when the pod re-created, we need to fail safe when loading session fail ?

controlplane/rosa/api/v1beta2/rosacontrolplane_types.go

controlplane/rosa/api/v1beta2/rosacontrolplane_webhook.go

damdo · 2025-09-30T10:19:52Z

controlplane/rosa/controllers/rosacontrolplane_controller.go

+		// Is the referenced ROSANetwork ready yet?
+		if !conditions.IsTrue(rosaNet, expinfrav1.ROSANetworkReadyCondition) {
+			rosaScope.Info(fmt.Sprintf("referenced ROSANetwork %s is not ready", rosaNet.Name))
+			return ctrl.Result{RequeueAfter: time.Minute}, nil


A minute seems quite a lot here, no?

It takes about 5 minutes to create the cloudformation stack, i.e. approximately 5 cycles through the reconciliation loop. I'm fine with making it smaller (suggestions welcome), but not quite sure how it would help or improve the situation.

Ok and do we need to requeue after or are we watching and we should get a reconciliation event anyway?

exp/controllers/rosanetwork_controller.go

damdo

/lgtm

Thanks for addressing my comments

/assign @nrb @richardcase

k8s-ci-robot · 2025-10-02T16:10:20Z

LGTM label has been added.

Git tree hash: 5f64d3fa3341e2ef3426645208e69a234b4147e3

damdo · 2025-10-06T09:27:53Z

Based on the convo at https://kubernetes.slack.com/archives/CD6U2V71N/p1759420550389709

/cherry-pick release-2.9

k8s-infra-cherrypick-robot · 2025-10-06T09:27:57Z

@damdo: once the present PR merges, I will cherry-pick it on top of release-2.9 in a new PR and assign it to you.

In response to this:

Based on the convo at https://kubernetes.slack.com/archives/CD6U2V71N/p1759420550389709

/cherry-pick release-2.9

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

richardcase · 2025-10-07T12:48:02Z

/approve

k8s-ci-robot · 2025-10-07T12:55:17Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: richardcase

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [richardcase]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

damdo · 2025-10-07T12:55:43Z

/cherry-pick release-2.9

k8s-infra-cherrypick-robot · 2025-10-07T12:55:45Z

@damdo: once the present PR merges, I will cherry-pick it on top of release-2.9 in a new PR and assign it to you.

In response to this:

/cherry-pick release-2.9

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

mzazrivec · 2025-10-07T15:54:36Z

/retest

damdo · 2025-10-07T16:14:25Z

Re-adding LGTM after rebase

/lgtm

k8s-ci-robot · 2025-10-07T16:14:32Z

LGTM label has been added.

Git tree hash: ebc14b0e8b4a2ee7c8be29ba1b22cc7b2fbbb814

k8s-infra-cherrypick-robot · 2025-10-07T16:19:40Z

@damdo: #5464 failed to apply on top of branch "release-2.9":

Applying: RosaNetwork: new CRD & reconciler to provision net infra for ROSA-HCP
Using index info to reconstruct a base tree...
M	PROJECT
M	config/crd/bases/controlplane.cluster.x-k8s.io_rosacontrolplanes.yaml
M	config/crd/kustomization.yaml
M	config/rbac/role.yaml
M	controlplane/rosa/api/v1beta2/rosacontrolplane_types.go
M	controlplane/rosa/api/v1beta2/rosacontrolplane_webhook.go
M	controlplane/rosa/api/v1beta2/zz_generated.deepcopy.go
M	controlplane/rosa/controllers/rosacontrolplane_controller.go
M	exp/api/v1beta2/zz_generated.deepcopy.go
M	go.mod
M	main.go
Falling back to patching base and 3-way merge...
Auto-merging main.go
Auto-merging go.mod
CONFLICT (content): Merge conflict in go.mod
Auto-merging exp/api/v1beta2/zz_generated.deepcopy.go
CONFLICT (content): Merge conflict in exp/api/v1beta2/zz_generated.deepcopy.go
Auto-merging controlplane/rosa/controllers/rosacontrolplane_controller.go
CONFLICT (content): Merge conflict in controlplane/rosa/controllers/rosacontrolplane_controller.go
Auto-merging controlplane/rosa/api/v1beta2/zz_generated.deepcopy.go
CONFLICT (content): Merge conflict in controlplane/rosa/api/v1beta2/zz_generated.deepcopy.go
Auto-merging controlplane/rosa/api/v1beta2/rosacontrolplane_webhook.go
CONFLICT (content): Merge conflict in controlplane/rosa/api/v1beta2/rosacontrolplane_webhook.go
Auto-merging controlplane/rosa/api/v1beta2/rosacontrolplane_types.go
CONFLICT (content): Merge conflict in controlplane/rosa/api/v1beta2/rosacontrolplane_types.go
Auto-merging config/rbac/role.yaml
CONFLICT (content): Merge conflict in config/rbac/role.yaml
Auto-merging config/crd/kustomization.yaml
CONFLICT (content): Merge conflict in config/crd/kustomization.yaml
Auto-merging config/crd/bases/controlplane.cluster.x-k8s.io_rosacontrolplanes.yaml
CONFLICT (content): Merge conflict in config/crd/bases/controlplane.cluster.x-k8s.io_rosacontrolplanes.yaml
Auto-merging PROJECT
CONFLICT (content): Merge conflict in PROJECT
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config advice.mergeConflict false"
Patch failed at 0001 RosaNetwork: new CRD & reconciler to provision net infra for ROSA-HCP

In response to this:

/cherry-pick release-2.9

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

serngawy · 2025-10-07T20:42:01Z

/cherry-pick release-2.9

k8s-infra-cherrypick-robot · 2025-10-07T20:42:38Z

@serngawy: new pull request created: #5701

In response to this:

/cherry-pick release-2.9

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot requested review from faiq and serngawy April 9, 2025 19:27

k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Apr 9, 2025

mzazrivec force-pushed the rosa_network branch from 3435873 to 22dea0b Compare April 9, 2025 19:32

serngawy reviewed Apr 10, 2025

View reviewed changes

mzazrivec force-pushed the rosa_network branch 4 times, most recently from 5907fb1 to 24a5950 Compare April 24, 2025 13:20

mzazrivec force-pushed the rosa_network branch 3 times, most recently from a947563 to a255790 Compare May 19, 2025 13:43

mzazrivec force-pushed the rosa_network branch from a255790 to d923d6f Compare May 28, 2025 12:17

serngawy reviewed May 28, 2025

View reviewed changes

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels May 28, 2025

mzazrivec force-pushed the rosa_network branch 4 times, most recently from d2534a7 to dcc599d Compare June 9, 2025 08:19

mzazrivec force-pushed the rosa_network branch 2 times, most recently from 023f99b to 1729dff Compare June 27, 2025 12:31

damdo reviewed Sep 30, 2025

View reviewed changes

mzazrivec force-pushed the rosa_network branch from bd3b722 to 974568c Compare October 1, 2025 13:47

damdo reviewed Oct 2, 2025

View reviewed changes

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 2, 2025

serngawy approved these changes Oct 7, 2025

View reviewed changes

damdo added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 7, 2025

mzazrivec added 2 commits October 7, 2025 17:36

RosaNetwork: new CRD & reconciler to provision net infra for ROSA-HCP

09397fd

ROSANetwork: tests

6c9c93e

mzazrivec force-pushed the rosa_network branch from 974568c to 6c9c93e Compare October 7, 2025 15:39

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 7, 2025

k8s-ci-robot requested review from damdo, nrb and richardcase October 7, 2025 15:39

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 7, 2025

k8s-ci-robot merged commit 516f342 into kubernetes-sigs:main Oct 7, 2025
17 checks passed

k8s-infra-cherrypick-robot mentioned this pull request Oct 7, 2025

[release-2.9] ✨ ROSANetwork: new CRD & reconciler to provision network infrastructure for ROSA-HCP #5701

Closed

serngawy mentioned this pull request Oct 8, 2025

[release-2.9] ✨ ROSANetwork: new CRD & reconciler to provision network infrastructure for ROSA-HCP #5702

Merged

5 tasks

	Reason string `json:"reason"`
	Message string `json:"message"`

	// ID of the public subnet
	// Public subnet Id ex; subnet-xxxxxxxxxx

✨ ROSANetwork: new CRD & reconciler to provision network infrastructure for ROSA-HCP #5464

✨ ROSANetwork: new CRD & reconciler to provision network infrastructure for ROSA-HCP #5464

Uh oh!

Conversation

mzazrivec commented Apr 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

k8s-ci-robot commented Apr 9, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

serngawy commented Apr 23, 2025

Uh oh!

serngawy left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

serngawy Aug 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

damdo left a comment

Choose a reason for hiding this comment

Uh oh!

k8s-ci-robot commented Oct 2, 2025

Uh oh!

damdo commented Oct 6, 2025

Uh oh!

k8s-infra-cherrypick-robot commented Oct 6, 2025

Uh oh!

richardcase commented Oct 7, 2025 • edited by damdo Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

k8s-ci-robot commented Oct 7, 2025

Uh oh!

damdo commented Oct 7, 2025

Uh oh!

mzazrivec commented Apr 9, 2025 •

edited

Loading

serngawy Aug 18, 2025 •

edited

Loading

richardcase commented Oct 7, 2025 •

edited by damdo

Loading