From 13a6a1662d484b652e8ba6207a5cf24297ea8689 Mon Sep 17 00:00:00 2001 From: Jim Enright Date: Mon, 29 Sep 2025 18:06:48 +0100 Subject: [PATCH 1/2] Add support for pre-existing resource group (but net new vnet) Signed-off-by: Jim Enright --- .../terraform-cdp-azure-pre-reqs/README.md | 11 +++++----- modules/terraform-cdp-azure-pre-reqs/main.tf | 20 +++++++++---------- .../terraform-cdp-azure-pre-reqs/variables.tf | 18 ++++++++++++----- 3 files changed, 29 insertions(+), 20 deletions(-) diff --git a/modules/terraform-cdp-azure-pre-reqs/README.md b/modules/terraform-cdp-azure-pre-reqs/README.md index c8235a3..61b7ac6 100644 --- a/modules/terraform-cdp-azure-pre-reqs/README.md +++ b/modules/terraform-cdp-azure-pre-reqs/README.md @@ -65,9 +65,9 @@ In each directory an example `terraform.tfvars.sample` values file is included t | [backup\_storage](#input\_backup\_storage) | Optional Backup location for CDP environment. If not provided follow the data\_storage variable | 
object({
    backup_storage_bucket = string
    backup_storage_object = string
  })
| `null` | no | | [cdp\_delegated\_subnet\_names](#input\_cdp\_delegated\_subnet\_names) | List of subnet names delegated for Flexible Servers. Required if create\_vnet is false. | `list(any)` | `null` | no | | [cdp\_gw\_subnet\_names](#input\_cdp\_gw\_subnet\_names) | List of subnet names for CDP Gateway. Required if create\_vnet is false. | `list(any)` | `null` | no | -| [cdp\_resourcegroup\_name](#input\_cdp\_resourcegroup\_name) | Resource Group name for resources. If create\_vnet is false this is a pre-existing resource group. | `string` | `null` | no | +| [cdp\_resourcegroup\_name](#input\_cdp\_resourcegroup\_name) | Resource Group name for resources. If either create\_vnet or create\_resource\_group is false this is a pre-existing resource group. | `string` | `null` | no | | [cdp\_subnet\_names](#input\_cdp\_subnet\_names) | List of subnet names for CDP Resources. Required if create\_vnet is false. | `list(any)` | `null` | no | -| [cdp\_subnet\_range](#input\_cdp\_subnet\_range) | Size of each (internal) cluster subnet. Required if create\_vpc is true. | `number` | `19` | no | +| [cdp\_subnet\_range](#input\_cdp\_subnet\_range) | Size of each (internal) cluster subnet. Required if create\_vnet is true. | `number` | `19` | no | | [cdp\_subnets\_private\_endpoint\_network\_policies](#input\_cdp\_subnets\_private\_endpoint\_network\_policies) | Enable or Disable network policies for the private endpoint on the CDP subnets | `string` | `"Enabled"` | no | | [cdp\_vnet\_name](#input\_cdp\_vnet\_name) | Pre-existing VNet Name for CDP environment. Required if create\_vnet is false. | `string` | `null` | no | | [create\_azure\_cml\_nfs](#input\_create\_azure\_cml\_nfs) | Whether to create NFS for CML | `bool` | `false` | no | @@ -75,6 +75,7 @@ In each directory an example `terraform.tfvars.sample` values file is included t | [create\_azure\_storage\_private\_endpoints](#input\_create\_azure\_storage\_private\_endpoints) | Flag to specify if Private Endpoints are created for each storage account. | `bool` | `true` | no | | [create\_nat\_gateway](#input\_create\_nat\_gateway) | Flag to specify if the NAT Gateway should be created. Only applicable if create\_vnet is true. | `bool` | `true` | no | | [create\_private\_flexible\_server\_resources](#input\_create\_private\_flexible\_server\_resources) | Flag to specify if resources to support a Private Postgres flexible server should be created. | `bool` | `null` | no | +| [create\_resource\_group](#input\_create\_resource\_group) | Flag to specify if the Resource Group should be created | `bool` | `true` | no | | [create\_vm\_mounting\_nfs](#input\_create\_vm\_mounting\_nfs) | Whether to create a VM which mounts this NFS | `bool` | `true` | no | | [create\_vnet](#input\_create\_vnet) | Flag to specify if the VNet should be created | `bool` | `true` | no | | [data\_storage](#input\_data\_storage) | Data storage locations for CDP environment | 
object({
    data_storage_bucket = string
    data_storage_object = string
  })
| `null` | no | @@ -82,14 +83,14 @@ In each directory an example `terraform.tfvars.sample` values file is included t | [datalake\_admin\_data\_container\_role\_assignments](#input\_datalake\_admin\_data\_container\_role\_assignments) | List of Role Assignments for the Datalake Admin Managed Identity assigned to the Data Storage Container. | 
list(object({
    role        = string
    description = string
    })
  )
| 
[
  {
    "description": "Assign Storage Blob Data Owner Role to Data Lake Admin Identity at Data Container Level",
    "role": "Storage Blob Data Owner"
  }
]
| no | | [datalake\_admin\_log\_container\_role\_assignments](#input\_datalake\_admin\_log\_container\_role\_assignments) | List of Role Assignments for the Datalake Admin Managed Identity assigned to the Logs Storage Container. | 
list(object({
    role        = string
    description = string
    })
  )
| 
[
  {
    "description": "Assign Storage Blob Data Owner Role to Data Lake Admin Identity at Logs Container Level",
    "role": "Storage Blob Data Owner"
  }
]
| no | | [datalake\_admin\_managed\_identity\_name](#input\_datalake\_admin\_managed\_identity\_name) | Datalake Admin Managed Identity name | `string` | `null` | no | -| [delegated\_subnet\_range](#input\_delegated\_subnet\_range) | Size of each Postgres Flexible Server delegated subnet. Required if create\_vpc is true. | `number` | `26` | no | +| [delegated\_subnet\_range](#input\_delegated\_subnet\_range) | Size of each Postgres Flexible Server delegated subnet. Required if create\_vnet is true. | `number` | `26` | no | | [enable\_raz](#input\_enable\_raz) | Flag to enable Ranger Authorization Service (RAZ) | `bool` | `true` | no | | [env\_tags](#input\_env\_tags) | Tags applied to provisioned resources | `map(any)` | `null` | no | | [existing\_default\_security\_group\_name](#input\_existing\_default\_security\_group\_name) | Name of existing Default Security Group for Cloudera on cloud environment. If set then no security group or ingress rules are created for the Default SG. | `string` | `null` | no | | [existing\_knox\_security\_group\_name](#input\_existing\_knox\_security\_group\_name) | Name of existing Knox Security Group for Cloudera on cloud environment. If set then no security group or ingress rules are created for the Knox SG. | `string` | `null` | no | | [existing\_xaccount\_app\_client\_id](#input\_existing\_xaccount\_app\_client\_id) | Client ID of existing Azure AD Application for Cloudera Cross Account. If set then no application or SPN resources are created. | `string` | `null` | no | | [existing\_xaccount\_app\_pword](#input\_existing\_xaccount\_app\_pword) | Password of existing Azure AD Application for Cloudera Cross Account. If set then no application or SPN resources are created. | `string` | `null` | no | -| [gateway\_subnet\_range](#input\_gateway\_subnet\_range) | Size of each gateway subnet. Required if create\_vpc is true. | `number` | `24` | no | +| [gateway\_subnet\_range](#input\_gateway\_subnet\_range) | Size of each gateway subnet. Required if create\_vnet is true. | `number` | `24` | no | | [gateway\_subnets\_private\_endpoint\_network\_policies](#input\_gateway\_subnets\_private\_endpoint\_network\_policies) | Enable or Disable network policies for the private endpoint on the Gateway subnets | `string` | `"Enabled"` | no | | [idbroker\_managed\_identity\_name](#input\_idbroker\_managed\_identity\_name) | IDBroker Managed Identity name | `string` | `null` | no | | [idbroker\_role\_assignments](#input\_idbroker\_role\_assignments) | List of Role Assignments for the IDBroker Managed Identity | 
list(object({
    role        = string
    description = string
    })
  )
| 
[
  {
    "description": "Assign VM Contributor Role to IDBroker Identity at Subscription Level",
    "role": "Virtual Machine Contributor"
  },
  {
    "description": "Assign Managed Identity Operator Role to IDBroker Identity at Subscription Level",
    "role": "Managed Identity Operator"
  }
]
| no | @@ -116,7 +117,7 @@ In each directory an example `terraform.tfvars.sample` values file is included t | [separate\_network\_resource\_group](#input\_separate\_network\_resource\_group) | Flag to specify if separate resource group is to be used for network and Cloudera resources | `bool` | `false` | no | | [storage\_public\_network\_access\_enabled](#input\_storage\_public\_network\_access\_enabled) | Enable public\_network\_access\_enabled for storage accounts. | `bool` | `true` | no | | [subnet\_count](#input\_subnet\_count) | Number of CDP Subnets Required | `string` | `"3"` | no | -| [vnet\_cidr](#input\_vnet\_cidr) | VNet CIDR Block. Required if create\_vpc is true. | `string` | `"10.10.0.0/16"` | no | +| [vnet\_cidr](#input\_vnet\_cidr) | VNet CIDR Block. Required if create\_vnet is true. | `string` | `"10.10.0.0/16"` | no | | [vnet\_name](#input\_vnet\_name) | VNet name | `string` | `null` | no | | [xaccount\_app\_name](#input\_xaccount\_app\_name) | Cross account application name within Azure Active Directory | `string` | `null` | no | | [xaccount\_app\_role\_assignments](#input\_xaccount\_app\_role\_assignments) | List of Role Assignments for the Cross Account Service Principal. If scope is not specified then scope is set to var.azure\_subscription\_id | 
list(object({
    role        = string
    description = string
    scope       = optional(string)
    })
  )
| 
[
  {
    "description": "Contributor Role to Cross Account Service Principal at Subscription Level",
    "role": "Contributor"
  }
]
| no | diff --git a/modules/terraform-cdp-azure-pre-reqs/main.tf b/modules/terraform-cdp-azure-pre-reqs/main.tf index 1474239..838a8d6 100644 --- a/modules/terraform-cdp-azure-pre-reqs/main.tf +++ b/modules/terraform-cdp-azure-pre-reqs/main.tf @@ -19,15 +19,15 @@ module "azure_cdp_rmgp" { source = "../terraform-azure-resource-group" - create_resource_group = var.create_vnet + create_resource_group = (var.create_resource_group && var.create_vnet) # Variables required when creating RG - resourcegroup_name = var.create_vnet ? local.cdp_resourcegroup_name : null - azure_region = var.create_vnet ? var.azure_region : null - tags = var.create_vnet ? var.env_tags : null + resourcegroup_name = (var.create_resource_group && var.create_vnet) ? local.cdp_resourcegroup_name : null + azure_region = (var.create_resource_group && var.create_vnet) ? var.azure_region : null + tags = (var.create_resource_group && var.create_vnet) ? var.env_tags : null # Variables required when using pre-existing RG - existing_resource_group_name = var.create_vnet ? null : var.cdp_resourcegroup_name + existing_resource_group_name = (var.create_resource_group && var.create_vnet) ? null : var.cdp_resourcegroup_name } @@ -37,15 +37,15 @@ module "azure_network_rmgp" { source = "../terraform-azure-resource-group" - create_resource_group = var.create_vnet + create_resource_group = (var.create_resource_group && var.create_vnet) # Variables required when creating RG - resourcegroup_name = var.create_vnet ? local.network_resourcegroup_name : null - azure_region = var.create_vnet ? var.azure_region : null - tags = var.create_vnet ? var.env_tags : null + resourcegroup_name = (var.create_resource_group && var.create_vnet) ? local.network_resourcegroup_name : null + azure_region = (var.create_resource_group && var.create_vnet) ? var.azure_region : null + tags = (var.create_resource_group && var.create_vnet) ? var.env_tags : null # Variables required when using pre-existing RG - existing_resource_group_name = var.create_vnet ? null : var.network_resourcegroup_name + existing_resource_group_name = (var.create_resource_group && var.create_vnet) ? null : var.network_resourcegroup_name } diff --git a/modules/terraform-cdp-azure-pre-reqs/variables.tf b/modules/terraform-cdp-azure-pre-reqs/variables.tf index 2c534fe..8d56b8f 100644 --- a/modules/terraform-cdp-azure-pre-reqs/variables.tf +++ b/modules/terraform-cdp-azure-pre-reqs/variables.tf @@ -92,7 +92,7 @@ variable "network_resourcegroup_name" { variable "cdp_resourcegroup_name" { type = string - description = "Resource Group name for resources. If create_vnet is false this is a pre-existing resource group." + description = "Resource Group name for resources. If either create_vnet or create_resource_group is false this is a pre-existing resource group." default = null @@ -108,6 +108,14 @@ variable "cdp_resourcegroup_name" { } +variable "create_resource_group" { + type = bool + + description = "Flag to specify if the Resource Group should be created" + + default = true +} + variable "create_vnet" { type = bool @@ -135,7 +143,7 @@ variable "vnet_name" { variable "vnet_cidr" { type = string - description = "VNet CIDR Block. Required if create_vpc is true." + description = "VNet CIDR Block. Required if create_vnet is true." default = "10.10.0.0/16" } @@ -164,21 +172,21 @@ variable "nat_public_ip_name" { variable "cdp_subnet_range" { type = number - description = "Size of each (internal) cluster subnet. Required if create_vpc is true." + description = "Size of each (internal) cluster subnet. Required if create_vnet is true." default = 19 } variable "gateway_subnet_range" { type = number - description = "Size of each gateway subnet. Required if create_vpc is true." + description = "Size of each gateway subnet. Required if create_vnet is true." default = 24 } variable "delegated_subnet_range" { type = number - description = "Size of each Postgres Flexible Server delegated subnet. Required if create_vpc is true." + description = "Size of each Postgres Flexible Server delegated subnet. Required if create_vnet is true." default = 26 } From 10efa33b1aa5f1a644f2359662b018c9204b313a Mon Sep 17 00:00:00 2001 From: Jim Enright Date: Tue, 30 Sep 2025 17:13:48 +0100 Subject: [PATCH 2/2] Add example of existing resource group to azure pre-reqs module Signed-off-by: Jim Enright --- .../terraform-cdp-azure-pre-reqs/README.md | 2 + .../doc_fragments/header.md | 2 + .../examples/ex02-existing-rg/main.tf | 52 +++++++++++++++++++ .../ex02-existing-rg/terraform.tfvars.sample | 29 +++++++++++ .../examples/ex02-existing-rg/variables.tf | 40 ++++++++++++++ 5 files changed, 125 insertions(+) create mode 100644 modules/terraform-cdp-azure-pre-reqs/examples/ex02-existing-rg/main.tf create mode 100644 modules/terraform-cdp-azure-pre-reqs/examples/ex02-existing-rg/terraform.tfvars.sample create mode 100644 modules/terraform-cdp-azure-pre-reqs/examples/ex02-existing-rg/variables.tf diff --git a/modules/terraform-cdp-azure-pre-reqs/README.md b/modules/terraform-cdp-azure-pre-reqs/README.md index 61b7ac6..4d38755 100644 --- a/modules/terraform-cdp-azure-pre-reqs/README.md +++ b/modules/terraform-cdp-azure-pre-reqs/README.md @@ -9,6 +9,8 @@ The [examples](./examples) directory has example Azure Cloud Service Provider de * `ex01-minimal-inputs` uses the minimum set of inputs for the module. +* `ex02-existing-rg` uses a pre-existing Azure resource group. + In each directory an example `terraform.tfvars.sample` values file is included to show input variable values. ## Requirements diff --git a/modules/terraform-cdp-azure-pre-reqs/doc_fragments/header.md b/modules/terraform-cdp-azure-pre-reqs/doc_fragments/header.md index bc711e0..7ec9b89 100644 --- a/modules/terraform-cdp-azure-pre-reqs/doc_fragments/header.md +++ b/modules/terraform-cdp-azure-pre-reqs/doc_fragments/header.md @@ -8,4 +8,6 @@ The [examples](./examples) directory has example Azure Cloud Service Provider de * `ex01-minimal-inputs` uses the minimum set of inputs for the module. +* `ex02-existing-rg` uses a pre-existing Azure resource group. + In each directory an example `terraform.tfvars.sample` values file is included to show input variable values. diff --git a/modules/terraform-cdp-azure-pre-reqs/examples/ex02-existing-rg/main.tf b/modules/terraform-cdp-azure-pre-reqs/examples/ex02-existing-rg/main.tf new file mode 100644 index 0000000..9d4a0e5 --- /dev/null +++ b/modules/terraform-cdp-azure-pre-reqs/examples/ex02-existing-rg/main.tf @@ -0,0 +1,52 @@ +# Copyright 2023 Cloudera, Inc. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +provider "azurerm" { + features { + resource_group { + prevent_deletion_if_contains_resources = false + } + } + +} + +provider "azuread" { + +} + +# ------- Azure Resource Group ------- +resource "azurerm_resource_group" "rmgp" { + + name = "${var.env_prefix}-existing-rg" + location = var.azure_region + + tags = { Name = "${var.env_prefix}-existing-rg" } +} + +module "ex04_existing_rg" { + source = "../.." + + env_prefix = var.env_prefix + azure_region = var.azure_region + + deployment_template = var.deployment_template + + ingress_extra_cidrs_and_ports = var.ingress_extra_cidrs_and_ports + + # Used existing resource group + create_resource_group = false + cdp_resourcegroup_name = azurerm_resource_group.rmgp.name + + depends_on = [azurerm_resource_group.rmgp] +} diff --git a/modules/terraform-cdp-azure-pre-reqs/examples/ex02-existing-rg/terraform.tfvars.sample b/modules/terraform-cdp-azure-pre-reqs/examples/ex02-existing-rg/terraform.tfvars.sample new file mode 100644 index 0000000..c3d5f25 --- /dev/null +++ b/modules/terraform-cdp-azure-pre-reqs/examples/ex02-existing-rg/terraform.tfvars.sample @@ -0,0 +1,29 @@ +# Copyright 2023 Cloudera, Inc. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# ------- Global settings ------- +env_prefix = "" # Required name prefix for cloud and CDP resources, e.g. cldr1 + +# ------- Cloud Settings ------- +azure_region = "" # Change this to specify Cloud Provider region, e.g. eastus + +# ------- CDP Environment Deployment ------- +deployment_template = "" # Specify the deployment pattern below. Options are public, semi-private or private + +# ------- Network Settings ------- +# **NOTE: If required change the values below any additional CIDRs to add the the AWS Security Groups** +ingress_extra_cidrs_and_ports = { + cidrs = ["/32", "/32"], + ports = [443, 22] +} \ No newline at end of file diff --git a/modules/terraform-cdp-azure-pre-reqs/examples/ex02-existing-rg/variables.tf b/modules/terraform-cdp-azure-pre-reqs/examples/ex02-existing-rg/variables.tf new file mode 100644 index 0000000..618c4ff --- /dev/null +++ b/modules/terraform-cdp-azure-pre-reqs/examples/ex02-existing-rg/variables.tf @@ -0,0 +1,40 @@ +# Copyright 2023 Cloudera, Inc. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# ------- Global settings ------- +variable "env_prefix" { + type = string + description = "Shorthand name for the environment. Used in resource descriptions" +} + +variable "azure_region" { + type = string + description = "Region which Cloud resources will be created" +} + +# ------- CDP Environment Deployment ------- +variable "deployment_template" { + type = string + + description = "Deployment Pattern to use for Cloud resources and CDP" +} + +# ------- Network Resources ------- +variable "ingress_extra_cidrs_and_ports" { + type = object({ + cidrs = list(string) + ports = list(number) + }) + description = "List of extra CIDR blocks and ports to include in Security Group Ingress rules" +}