Add standalone modules for AWS and Azure ingress/security groups

README.md

@@ -12,6 +12,7 @@ This repository contains a number of Terraform modules for creation of the pre-r
 | [terraform-cdp-deploy](modules/terraform-cdp-deploy/README.md) | For deployment of CDP on AWS, Azure or GCP. |
 | [terraform-aws-cred-permissions](modules/terraform-aws-cred-permissions/README.md) | Module for creation of the Cross Account Credential pre-requisite on AWS. Note that this module is called from the terraform-cdp-aws-prereqs module. |
 | [terraform-aws-permissions](modules/terraform-aws-permissions/README.md) | Module for creation of the AWS IAM permissions required by the (CDP) Public Cloud environment and datalake deployment. Note that this module is called from the terraform-cdp-aws-prereqs module. |
+| [terraform-aws-ingress](modules/terraform-aws-ingress/README.md) | Module for creation and management of the Default and Knox AWS security groups for Cloudera on cloud deployments. Note that this module is called from the terraform-cdp-aws-prereqs module. |
 | [terraform-aws-vpc](modules/terraform-aws-vpc/README.md) | Module for creation of the VPC networking resources on AWS. Can be used to create the CDP VPC and Subnets. Note that this module is called from the terraform-cdp-aws-prereqs module. |
 | [terraform-aws-fw-vpc](modules/terraform-aws-fw-vpc/README.md) | Module for creation of the VPC networking resources on AWS suitable for running a Firewall in a distributed architecture on AWS. Can be used to create a networking VPC which runs the AWS Network Firewall and connects to a Cloudera on cloud full-private deployment. |
 | [terraform-aws-tgw](modules/terraform-aws-tgw/README.md) | Module for creation of AWS Transity Gateway (TGW) and attaching a specified list of VPCs via the TGW. This module can be used to assist in deploying Cloudera Data Platform (CDP) Public Cloud in a fully private networking configuration where a CDP VPC and Networking VPC are connected using the Transit Gateway. |
@@ -20,6 +21,7 @@ This repository contains a number of Terraform modules for creation of the pre-r
 | [terraform-aws-permissions](modules/terraform-aws-permissions/README.md) | Module for creation of the AWS IAM permissions required by the (CDP) Public Cloud environment and datalake deployment. Note that this module is called from the terraform-cdp-aws-prereqs module. |
 | [terraform-aws-firewall](modules/terraform-aws-firewall/README.md) | Module to create and configure to create and configure an AWS Network Firewall. This module can be used to assist in deploying Cloudera Data Platform (CDP) Public Cloud in a fully private networking configuration where the CDP Environment is connected to a Networking VPC running the Firewall. |
 | [terraform-azure-resource-group](modules/terraform-azure-resource-group/README.md) | Module for creation of a Resource Group on Azure. Can be used for creation of the pre-requisite resource group for Cloudera Data Platform (CDP) Public Cloud. |
+| [terraform-azure-ingress](modules/terraform-azure-ingress/README.md) | Module for creation and management of the Default and Knox Azure network security groups for Cloudera on cloud deployments. Note that this module is called from the terraform-cdp-azure-prereqs module. |
 | [terraform-azure-vnet](modules/terraform-azure-vnet/README.md) | Module for creation of the Virtual Network (VNET) on Azure. Can be used for creation of the pre-requisite VNet and subnets for Cloudera Data Platform (CDP) Public Cloud. |
 | [terraform-azure-cred-permissions](modules/terraform-azure-cred-permissions/README.md) | Module for creation of the Cross Account Credential pre-requisites on Azure. Note that this module is called from the terraform-cdp-azure-prereqs module. |
 | [terraform-azure-bastion](modules/terraform-azure-bastion/README.md) | Module to create a Bastion Virtual Machine instance on Azure. |

modules/terraform-aws-ingress/.terraform-docs.yaml

@@ -0,0 +1,21 @@
+formatter: markdown
+header-from: doc_fragments/header.md
+settings:
+  anchor: true
+  color: true
+  default: true
+  escape: true
+  html: true
+  indent: 2
+  required: true
+  sensitive: true
+  type: true
+
+
+sort:
+  enabled: true
+  by: required
+
+output:
+  file: README.md
+  mode: replace

modules/terraform-aws-ingress/README.md

@@ -0,0 +1,82 @@
+<!-- BEGIN_TF_DOCS -->
+# Terraform Module for AWS Ingress (Security Groups)
+
+This module contains resource files and example variable definition files for creating and managing the Default and Knox AWS security groups for Cloudera on cloud deployments.
+
+Support for using a pre-existing Security Groups is provided via the `existing_default_security_group_name` and `existing_knox_security_group_name` input variables. When this is set no security group resources are created. Instead a lookup of the details of the existing security group takes place and the ID is returned.
+
+## Usage
+
+The [examples](./examples) directory has example of using this module:
+
+* `ex01-minimal_inputs` demonstrates how this module can be used to create security groups with minimum required inputs. The [terraform-aws-vpc](../../../terraform-aws-vpc/README.md) module is also used as part of this example.
+* `ex02-existing_sgs` demonstrates how to use existing security groups with this module.
+
+The README and sample `terraform.tfvars.sample` describe how to use the example.
+
+## Requirements
+
+| Name | Version |
+|------|---------|
+| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | > 1.3.0 |
+| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 5.30 |
+
+## Providers
+
+| Name | Version |
+|------|---------|
+| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 5.30 |
+
+## Modules
+
+No modules.
+
+## Resources
+
+| Name | Type |
+|------|------|
+| [aws_ec2_managed_prefix_list.cdp_prefix_list](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ec2_managed_prefix_list) | resource |
+| [aws_ec2_managed_prefix_list_entry.cdp_prefix_list_entry](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ec2_managed_prefix_list_entry) | resource |
+| [aws_security_group.cdp_default_sg](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) | resource |
+| [aws_security_group.cdp_knox_sg](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) | resource |
+| [aws_vpc_security_group_egress_rule.cdp_default_sg_egress](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_security_group_egress_rule) | resource |
+| [aws_vpc_security_group_egress_rule.cdp_knox_sg_egress](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_security_group_egress_rule) | resource |
+| [aws_vpc_security_group_ingress_rule.cdp_default_extra_sg_ingress](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_security_group_ingress_rule) | resource |
+| [aws_vpc_security_group_ingress_rule.cdp_default_extra_sg_ingress_pl](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_security_group_ingress_rule) | resource |
+| [aws_vpc_security_group_ingress_rule.cdp_default_sg_ingress_self](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_security_group_ingress_rule) | resource |
+| [aws_vpc_security_group_ingress_rule.cdp_default_vpc_sg_ingress](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_security_group_ingress_rule) | resource |
+| [aws_vpc_security_group_ingress_rule.cdp_knox_extra_sg_ingress](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_security_group_ingress_rule) | resource |
+| [aws_vpc_security_group_ingress_rule.cdp_knox_extra_sg_ingress_pl](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_security_group_ingress_rule) | resource |
+| [aws_vpc_security_group_ingress_rule.cdp_knox_sg_ingress_self](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_security_group_ingress_rule) | resource |
+| [aws_vpc_security_group_ingress_rule.cdp_knox_vpc_sg_ingress](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_security_group_ingress_rule) | resource |
+| [aws_security_group.cdp_default_sg](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/security_group) | data source |
+| [aws_security_group.cdp_knox_sg](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/security_group) | data source |
+| [aws_vpc.vpc](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/vpc) | data source |
+
+## Inputs
+
+| Name | Description | Type | Default | Required |
+|------|-------------|------|---------|:--------:|
+| <a name="input_vpc_id"></a> [vpc\_id](#input\_vpc\_id) | VPC ID for where the security groups will be created. | `string` | n/a | yes |
+| <a name="input_cdp_default_sg_egress_cidrs"></a> [cdp\_default\_sg\_egress\_cidrs](#input\_cdp\_default\_sg\_egress\_cidrs) | List of egress CIDR blocks for CDP Default Security Group Egress rule | `list(string)` | <pre>[<br/>  "0.0.0.0/0"<br/>]</pre> | no |
+| <a name="input_cdp_knox_sg_egress_cidrs"></a> [cdp\_knox\_sg\_egress\_cidrs](#input\_cdp\_knox\_sg\_egress\_cidrs) | List of egress CIDR blocks for CDP Knox Security Group Egress rule | `list(string)` | <pre>[<br/>  "0.0.0.0/0"<br/>]</pre> | no |
+| <a name="input_default_security_group_name"></a> [default\_security\_group\_name](#input\_default\_security\_group\_name) | Default Security Group for Cloudera on cloud environment | `string` | `null` | no |
+| <a name="input_existing_default_security_group_name"></a> [existing\_default\_security\_group\_name](#input\_existing\_default\_security\_group\_name) | Name of existing Default Security Group for Cloudera on cloud environment. If set then no security group or ingress rules are created for the Default SG. | `string` | `null` | no |
+| <a name="input_existing_knox_security_group_name"></a> [existing\_knox\_security\_group\_name](#input\_existing\_knox\_security\_group\_name) | Name of existing Knox Security Group for Cloudera on cloud environment. If set then no security group or ingress rules are created for the Knox SG. | `string` | `null` | no |
+| <a name="input_ingress_extra_cidrs_and_ports"></a> [ingress\_extra\_cidrs\_and\_ports](#input\_ingress\_extra\_cidrs\_and\_ports) | List of extra CIDR blocks and ports to include in Security Group Ingress rules | <pre>object({<br/>    cidrs = list(string)<br/>    ports = list(number)<br/>  })</pre> | <pre>{<br/>  "cidrs": [],<br/>  "ports": []<br/>}</pre> | no |
+| <a name="input_ingress_vpc_cidr"></a> [ingress\_vpc\_cidr](#input\_ingress\_vpc\_cidr) | VPC CIDR block to include in Security Group Ingress rule | `string` | `null` | no |
+| <a name="input_knox_security_group_name"></a> [knox\_security\_group\_name](#input\_knox\_security\_group\_name) | Knox Security Group for Cloudera on cloud environment | `string` | `null` | no |
+| <a name="input_prefix_list_name"></a> [prefix\_list\_name](#input\_prefix\_list\_name) | Name of the AWS Prefix List to use for the security group rules. | `string` | `null` | no |
+| <a name="input_tags"></a> [tags](#input\_tags) | Tags applied to provisioned resources | `map(any)` | `null` | no |
+| <a name="input_use_prefix_list_for_ingress"></a> [use\_prefix\_list\_for\_ingress](#input\_use\_prefix\_list\_for\_ingress) | Whether to use prefix lists for ingress rules instead of direct CIDR blocks | `bool` | `true` | no |
+
+## Outputs
+
+| Name | Description |
+|------|-------------|
+| <a name="output_aws_prefix_list_id"></a> [aws\_prefix\_list\_id](#output\_aws\_prefix\_list\_id) | AWS managed prefix list ID |
+| <a name="output_aws_security_group_default_arn"></a> [aws\_security\_group\_default\_arn](#output\_aws\_security\_group\_default\_arn) | AWS security group ARN for default CDP SG |
+| <a name="output_aws_security_group_default_id"></a> [aws\_security\_group\_default\_id](#output\_aws\_security\_group\_default\_id) | AWS security group id for default CDP SG |
+| <a name="output_aws_security_group_knox_arn"></a> [aws\_security\_group\_knox\_arn](#output\_aws\_security\_group\_knox\_arn) | AWS security group ARN for Knox CDP SG |
+| <a name="output_aws_security_group_knox_id"></a> [aws\_security\_group\_knox\_id](#output\_aws\_security\_group\_knox\_id) | AWS security group id for Knox CDP SG |
+<!-- END_TF_DOCS -->

modules/terraform-aws-ingress/data.tf

@@ -0,0 +1,30 @@
+# Copyright 2025 Cloudera, Inc. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Find details of the VPC
+data "aws_vpc" "vpc" {
+  id = var.vpc_id
+}
+
+data "aws_security_group" "cdp_default_sg" {
+
+  name   = local.create_default_security_group ? aws_security_group.cdp_default_sg[0].name : var.existing_default_security_group_name
+  vpc_id = var.vpc_id
+}
+
+data "aws_security_group" "cdp_knox_sg" {
+
+  name   = local.create_knox_security_group ? aws_security_group.cdp_knox_sg[0].name : var.existing_knox_security_group_name
+  vpc_id = var.vpc_id
+}

modules/terraform-aws-ingress/defaults.tf

@@ -0,0 +1,32 @@
+# Copyright 2025 Cloudera, Inc. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+locals {
+
+  # ------- Determine if resources should be created -------
+  create_default_security_group = (var.existing_default_security_group_name == null)
+
+  create_knox_security_group = (var.existing_knox_security_group_name == null)
+
+  security_group_rules_extra_ingress = flatten([
+    for i, cidr in var.ingress_extra_cidrs_and_ports.cidrs : [
+      for j, port in var.ingress_extra_cidrs_and_ports.ports :
+      {
+        cidr     = cidr
+        port     = port
+        protocol = "tcp"
+    }]
+  ])
+
+}

modules/terraform-aws-ingress/doc_fragments/header.md

@@ -0,0 +1,14 @@
+# Terraform Module for AWS Ingress (Security Groups)
+
+This module contains resource files and example variable definition files for creating and managing the Default and Knox AWS security groups for Cloudera on cloud deployments.
+
+Support for using a pre-existing Security Groups is provided via the `existing_default_security_group_name` and `existing_knox_security_group_name` input variables. When this is set no security group resources are created. Instead a lookup of the details of the existing security group takes place and the ID is returned.
+
+## Usage
+
+The [examples](./examples) directory has example of using this module:
+
+* `ex01-minimal_inputs` demonstrates how this module can be used to create security groups with minimum required inputs. The [terraform-aws-vpc](../../../terraform-aws-vpc/README.md) module is also used as part of this example.
+* `ex02-existing_sgs` demonstrates how to use existing security groups with this module.
+
+The README and sample `terraform.tfvars.sample` describe how to use the example.

modules/terraform-aws-ingress/examples/ex01-minimal_inputs/main.tf

@@ -0,0 +1,50 @@
+# Copyright 2025 Cloudera, Inc. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+provider "aws" {
+  region = var.aws_region
+}
+
+module "ex01_vpc" {
+  source = "../../../terraform-aws-vpc"
+
+  cdp_vpc            = false
+  vpc_name           = "${var.name_prefix}-vpc"
+  vpc_cidr           = "10.11.0.0/16"
+  enable_nat_gateway = false
+
+  private_cidr_range = 26
+  public_cidr_range  = 26
+}
+
+module "ex01_sg" {
+  source = "../.."
+
+  vpc_id = module.ex01_vpc.vpc_id
+
+  default_security_group_name = "${var.name_prefix}-default-sg"
+  knox_security_group_name    = "${var.name_prefix}-knox-sg"
+  prefix_list_name            = "${var.name_prefix}-prefix-list"
+  # use_prefix_list_for_ingress = false
+
+  ingress_vpc_cidr = module.ex01_vpc.vpc_cidr_blocks[0]
+
+  ingress_extra_cidrs_and_ports = var.ingress_extra_cidrs_and_ports
+
+
+  tags = var.tags
+
+  depends_on = [module.ex01_vpc]
+
+}

modules/terraform-aws-ingress/examples/ex01-minimal_inputs/outputs.tf

@@ -0,0 +1,37 @@
+# Copyright 2025 Cloudera, Inc. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# output "bastion_instance_public_ip" {
+#   value = module.ex01_bastion.bastion_instance_public_ip
+
+#   description = "The public IP assigned to the Bastion host."
+# }
+
+# output "bastion_instance_private_ip" {
+#   value = module.ex01_bastion.bastion_instance_private_ip
+
+#   description = "The private IP assigned to the Bastion host."
+# }
+
+# output "bastion_instance_details" {
+#   value = module.ex01_bastion.bastion_instance_details
+
+#   description = "The details of the Bastion host."
+# }
+
+# output "bastion_instance_id" {
+#   value = module.ex01_bastion.bastion_instance_id
+
+#   description = "The instance ID assigned to the Bastion host."
+# }

modules/terraform-aws-ingress/examples/ex01-minimal_inputs/terraform.tfvars.sample

@@ -0,0 +1,30 @@
+# Copyright 2025 Cloudera, Inc. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# ------- Global Settings -------
+name_prefix = "<ENTER_VALUE>"
+
+tags = {
+  "Project" = "terraform-cdp-modules"
+  "Module"  = "terraform-aws-ingress"
+}
+
+# ------- Cloud Settings -------
+aws_region = "<ENTER_VALUE>" # Change this to specify Cloud Provider region, e.g. eu-west-1
+
+# ------- Ingress settings -------
+ingress_extra_cidrs_and_ports = {
+ cidrs = ["<ENTER_IP_VALUE>/32", "<ENTER_IP_VALUE>/32"],
+ ports = [443, 22]
+}