- Description
- How to release
- Breaking changes
- Supported versions
- Examples
- Datadog
- Amazon EFS CSI driver
- Terraform tips and tricks
- Requirements
- Providers
- Modules
- Resources
- Inputs
- Outputs
This module is used to create an EKS cluster on AWS with the support of the vpc
and sso-roles modules.
New releases are created automagically by Release Drafter GH action.
Type of release bump is made of commits (tags feat/bugfix/etc...).
Release is created as draft, so you have to edit it manually and change it to final.
The Traefik external ALB, internal NLB and their Kubernetes namespaces, Services,
Ingress and node security group rules have been removed. Gateway API load balancers
(gateway_api_*) are the only load balancers managed by the module.
Removed inputs (delete them from the module call when bumping):
external_alb_enabled, internal_nlb_enabled, traefik_cert_arn (use external_cert_arn),
traefik_nlb_service_ports, internal_nlb_service_ports, extra_nlb_listeners,
alb_additional_node_ports, acm_extra_arns, wafv2_arn (attach WAF to
gateway_api_external_alb_arn instead).
Removed outputs: alb_dns_name, alb_arn, alb_dns_names, alb_arns,
nlb_dns_names, nlb_zone_ids, nlb_arns. Use the corresponding gateway_api_*
DNS name and ARN outputs for Gateway API load balancers. There is no Gateway API
NLB zone-ID output. For a Route 53 alias, look up the Gateway NLB by its ARN with
the aws_lb data source and use its zone_id.
Migration: clusters must already run with external_alb_enabled = false and
internal_nlb_enabled = false and have no Traefik load balancers in state. On such
clusters the bump plans no Traefik changes. A cluster that still has a Traefik ALB/NLB
must tear it down on a 11.x release first; bumping directly would destroy it.
The controller role allows ec2:DescribeInstance* rather than only the four
previously enumerated instance-description actions. This includes the
instance-description APIs needed by newer Karpenter controllers. The existing
iam:ListInstanceProfiles permission remains unchanged. Apply and verify the
updated controller IAM policy on each cluster before upgrading its Karpenter
chart; a module release alone does not change deployed permissions.
The spot interruption EventBridge rules (aws_cloudwatch_event_rule.spot_aws_health
and aws_cloudwatch_event_rule.spot_aws_ec2) are now named per-cluster:
spot-aws-health-${cluster_name} and spot-aws-ec2-${cluster_name}. Previously
they used the fixed regional names spot-aws-health / spot-aws-ec2, which two
clusters in the same AWS account and region shared and fought over from separate
Terraform states.
Migration for existing clusters: the Terraform resource addresses are unchanged
(only the name attribute changed), so no state surgery is required — a normal
terraform apply replaces each rule and its target in place. EventBridge rule
replacement is a destroy-then-create; expect a brief (few-second) window during the
apply where spot interruption events for that cluster are not delivered to its SQS
queue. Run the apply during a maintenance window if that matters for the cluster.
After migration, each cluster owns its own rules, so creating or destroying one
cluster no longer affects spot interruption handling for another cluster in the same
account/region. The previous manual terraform state rm workaround during cluster
removal is no longer needed and has been removed from the removal steps below.
Version 10.0.0 introduces a breaking change due to upgrading deprecated kubernetes_* resources to their kubernetes_*_v1 counterparts. This affects the following resources:
kubernetes_namespace→kubernetes_namespace_v1kubernetes_service→kubernetes_service_v1kubernetes_secret→kubernetes_secret_v1kubernetes_storage_class→kubernetes_storage_class_v1
Why moved blocks don't work:
Terraform's moved block cannot be used for this migration because the Kubernetes provider does not support moving resource state across different resource types:
Error: Move Resource State Not Supported
The "kubernetes_storage_class_v1" resource type does not support moving resource state across resource types.
Suggested Solution:
Add the following removed and import blocks to your root module (e.g., state.tf) to migrate resources without destroying them:
# Remove old resource from state without destroying it
removed {
from = module.eks_security.kubernetes_storage_class.gp3
lifecycle {
destroy = false
}
}
# Import existing storage class into new resource type
import {
to = module.eks_security.kubernetes_storage_class_v1.gp3[0]
id = "gp3"
}After successful migration (
terraform apply), you can remove theremovedandimportblocks from your configuration.
Version 4.0 introduces an authentication mode change from CONFIG_MAP to API_AND_CONFIG_MAP. This change requires manual intervention to update the clusters. The following steps should be taken to update the clusters:
aws eks update-cluster-config --name CLUSTER_NAME --access-config authenticationMode=API_AND_CONFIG_MAP --region AWS_REGION
This will change the authentication mode to API_AND_CONFIG_MAP, and the next terraform plan/apply will work as expected.
The module is currently supporting the following versions of Kubernetes:
- 1.32,
- 1.33,
- 1.34,
- 1.35,
- 1.36,
Note
Default version for EKS Cluster is 1.35.
A minimal example of how to use this module.
module "eks" {
source = "git@github.com:worldcoin/terraform-aws-eks?ref=v12.0.0"
cluster_name = local.cluster_name
region = var.region
vpc_config = module.vpc.config
extra_role_mapping = module.sso_roles.default_mappings
datadog_api_key = var.datadog_api_key
external_cert_arn = var.external_cert_arn
alb_logs_bucket_id = module.region.alb_logs_bucket_id
}Example of Internal load balancer setup
module "eks" {
source = "git@github.com:worldcoin/terraform-aws-eks?ref=v12.0.0"
cluster_name = local.cluster_name
region = var.region
vpc_config = module.vpc.config
extra_role_mapping = module.sso_roles.default_mappings
datadog_api_key = var.datadog_api_key
external_cert_arn = var.external_cert_arn
alb_logs_bucket_id = module.region.alb_logs_bucket_id
gateway_api_crds_enabled = true
gateway_api_internal_enabled = true
internal_cert_arn = module.acm.cert_arn
}Example off using Static Auto Scaling Group
module "eks" {
source = "git@github.com:worldcoin/terraform-aws-eks?ref=v12.0.0"
cluster_name = local.cluster_name
region = var.region
vpc_config = module.vpc.config
extra_role_mapping = module.sso_roles.default_mappings
environment = var.environment
external_cert_arn = module.acm_v3.cert_arn
datadog_api_key = var.datadog_api_key
alb_logs_bucket_id = module.region.alb_logs_bucket_id
monitoring_enabled = false
static_autoscaling_group = {
size = 8
arch = "arm64"
type = "m7g.16xlarge"
}
}Example of using private subnets for the internal Gateway API NLB
module "eks" {
source = "git@github.com:worldcoin/terraform-aws-eks?ref=v12.0.0"
cluster_name = local.cluster_name
region = var.region
vpc_config = module.vpc.config
extra_role_mapping = module.sso_roles.default_mappings
environment = var.environment
external_cert_arn = module.acm_v3.cert_arn
datadog_api_key = var.datadog_api_key
alb_logs_bucket_id = module.region.alb_logs_bucket_id
monitoring_enabled = false
gateway_api_crds_enabled = true
gateway_api_internal_enabled = true
use_private_subnets_for_internal_nlb = true
}Example of using additional_security_group_rules to add rules to the node security group and additional_cluster_security_group_rules for the cluster security group.
module "eks" {
source = "git@github.com:worldcoin/terraform-aws-eks?ref=v12.0.0"
cluster_name = local.cluster_name
region = var.region
environment = var.environment
vpc_config = module.vpc.config
extra_role_mapping = module.sso_roles.default_mappings
external_cert_arn = module.acm.cert_arn
datadog_api_key = var.datadog_api_key
alb_logs_bucket_id = module.region.alb_logs_bucket_id
monitoring_notification_channel = "@slack-TFH-infrastructure-alerts-stage"
# Add rules to the NODE security group
additional_security_group_rules = [
{
type = "ingress"
from_port = 0
to_port = 0
protocol = "-1"
description = "Allow all ingress traffic from specific CIDR blocks"
cidr_blocks = ["10.100.0.0/16", "192.168.0.0/24"]
},
{
type = "ingress"
from_port = 0
to_port = 0
protocol = "-1"
description = "Allow all ingress traffic from the cluster security group"
source_cluster_security_group = true
}
]
# Add rules to the CLUSTER security group
additional_cluster_security_group_rules = [
{
type = "ingress"
from_port = 0
to_port = 0
protocol = "-1"
description = "Allow all ingress traffic from the node security group"
source_node_security_group = true
}
]
}The access_entries input allows you to associate access policies with access entries. The access_entries input is a map where the key is the name of the access entry and the value is a map with the following keys:
module "eks" {
source = "git@github.com:worldcoin/terraform-aws-eks?ref=v12.0.0"
cluster_name = local.cluster_name
region = var.region
vpc_config = module.vpc.config
extra_role_mapping = module.sso_roles.default_mappings
datadog_api_key = var.datadog_api_key
external_cert_arn = var.external_cert_arn
alb_logs_bucket_id = module.region.alb_logs_bucket_id
access_entries = {
# example with cluster access with default AmazonEKSAdminPolicy
applicationA = {
principal_arn = "arn:aws:iam::507152310572:role/github-deployment-applicationA"
access_scope_type = "cluster"
}
# example with namespace access
applicationB = {
principal_arn = "arn:aws:iam::507152310572:role/github-deployment-applicationB"
access_scope_namespaces = ["applicationB"]
}
# example with policy AmazonEKSClusterAdminPolicy access
applicationC = {
principal_arn = "arn:aws:iam::507152310572:role/github-deployment-applicationC"
access_scope_type = "cluster"
policy_arn = "arn:aws:eks::aws:cluster-access-policy/AmazonEKSClusterAdminPolicy"
}
}
}Once the policy_arn is not specified, the default AmazonEKSAdminPolicy is used.
Most common used AWS policies for EKS clusters:
- AmazonEKSClusterAdminPolicy: This policy grants administrator access to a cluster and is equivalent to the RBAC cluster-admin role with star permissions on everything.
- AmazonEKSAdminPolicy: This policy is equivalent to the RBAC admin role. It provides broad permissions to resources, typically scoped to a specific namespace. It is somewhat restricted when it comes to modifying namespace configurations or affecting other namespaces. This policy is designed to support namespace-based multi-tenancy. If you want an IAM principal to have a more limited administrative scope, consider using AmazonEKSAdminPolicy instead of AmazonEKSClusterAdminPolicy.
- AmazonEKSEditPolicy: This policy grants access to edit most Kubernetes resources, usually within a specific namespace. It allows reading secrets and editing resources, but it should not serve as a security boundary, as there are several possible privilege escalation paths to AmazonEKSClusterAdminPolicy.
- AmazonEKSViewPolicy: Grants access to list and view most Kubernetes resources, typically within a namespace. This policy is read-only and does not allow modification of resources. It is useful for monitoring and auditing purposes.
In summary, AmazonEKSClusterAdminPolicy provides the highest level of access, while AmazonEKSAdminPolicy and AmazonEKSEditPolicy offer more restricted, namespace-scoped permissions.
If you need specyfic access to the cluster, you can list the available AWS Polivies via aws cli:
aws eks list-access-policies --output table --region us-east-1
The module is creating a DataDog integration secret for the apiKeyExistingSecret of the DataDog helm chart.
Example with basic enclave support:
module "eks" {
source = "git@github.com:worldcoin/terraform-aws-eks?ref=v7.11.0"
# ... other configuration ...
# Enable basic enclave support (legacy)
enclaves = true
enclaves_instance_type = "m7a.4xlarge"
enclaves_autoscaling_group = {
size = 2
min_size = 1
max_size = 4
}
enclaves_cpu_allocation = "8"
enclaves_memory_allocation = "8192"
}Example with enclave tracks for running multiple versions simultaneously:
module "eks" {
source = "git@github.com:worldcoin/terraform-aws-eks?ref=v7.11.0"
# ... other configuration ...
# Multiple enclave tracks for different versions
enclave_tracks = {
v1 = {
autoscaling_group = {
size = 3 # Spreads across AZs
min_size = 3
max_size = 9
}
instance_type = "m7a.4xlarge"
cpu_allocation = "8"
memory_allocation = "8192"
}
v2 = {
autoscaling_group = {
size = 1
min_size = 0
max_size = 3
}
instance_type = "m7a.2xlarge"
# Uses default cpu/memory allocation if not specified
}
}
}Each track creates:
- Dedicated ASG with nodes spread across availability zones
- Node labels:
enclave.tools/track=<track_name> - Node taints:
enclave.tools/track=<track_name>:NoSchedule
Deploy workloads to specific tracks using nodeSelector:
nodeSelector:
aws-nitro-enclaves-k8s-dp: enabled
enclave.tools/track: "stable"
tolerations:
- key: "enclave"
operator: "Exists"
effect: "NoExecute"For detailed enclave tracks documentation, see ENCLAVE_TRACKS.md.
Monitoring the cluster using Datadog is also included, enabled by default, by using terraform-datadog-kubernetes.
The module comes with the IAM role for Amazon EFS CSI driver, and can be enabled using efs_csi_driver_enabled variable. Also with the role, it will create an instance of Elastic File System (EFS) and mount it to the cluster as a StorageClass named efs.
The file system policy permits NFS client access through mount targets and requires TLS. It preserves POSIX/root access without requiring IAM-authenticated mounts; network access remains restricted to the EKS node security group. Existing clients must use TLS before upgrading to a module version with this policy. This policy does not replace application-level POSIX permissions or provide per-application IAM isolation.
The module manages Gateway API CRDs, GatewayClass, LoadBalancerConfiguration, and Gateway resources via kubernetes_manifest.
Due to a known provider limitation, kubernetes_manifest requires the API server to be reachable and CRDs to be registered during plan — not just apply.
This means enabling Gateway API on a new cluster requires 3 applies:
| Apply | What changes | What happens |
|---|---|---|
| 1 | kubernetes_provider_enabled = true |
Cluster + AWS infra + basic K8s resources (namespaces, storage class, aws-auth, etc.) |
| 2 | gateway_api_crds_enabled = true |
Gateway API + AWS LBC CRDs installed |
| 3 | gateway_api_external_enabled = true and/or gateway_api_internal_enabled = true |
GatewayClass + Gateway + LoadBalancerConfiguration |
Example (final state):
module "eks" {
source = "git@github.com:worldcoin/terraform-aws-eks?ref=vX.Y.Z"
cluster_name = "my-cluster-us-east-1"
region = "us-east-1"
environment = "dev"
vpc_config = module.vpc.config
extra_role_mapping = module.sso_roles.default_mappings
datadog_api_key = var.datadog_api_key
alb_logs_bucket_id = module.region.alb_logs_bucket_id
external_cert_arn = module.acm.cert_arn
internal_cert_arn = module.acm.cert_arn
gateway_api_crds_enabled = true
gateway_api_external_enabled = true
gateway_api_internal_enabled = true
}Migrating from ArgoCD Helm charts:
For clusters currently using the gateway-api-crds, gateway-api-external, and gateway-api-internal Helm charts, see imports-gateway-api.tf.example for import blocks that enable zero-downtime migration.
- Copy
imports-gateway-api.tf.exampleto your root module - Replace
module.ekswith your module name terraform apply— imports existing resources- Remove the gateway-api-* Helm charts from ArgoCD
- Delete the import file
Internal Gateway API load balancers allow 10.0.0.0/8 by default (ALB: TCP/443;
NLB: TCP/80 and TCP/443). Custom security group rules are additive. To replace
the broad IPv4 defaults, configure explicit client rules and disable the defaults
independently for each load balancer:
gateway_api_internal_alb_default_ipv4_ingress_enabled = false
gateway_api_internal_alb_sg_rules = [{
description = "Allow HTTPS from approved client VPC"
port = 443
cidr_blocks = ["10.0.160.0/20"]
}]
gateway_api_internal_nlb_default_ipv4_ingress_enabled = false
gateway_api_internal_nlb_sg_rules = [{
description = "Allow HTTPS from approved PrivateLink client VPC"
port = 443
cidr_blocks = ["10.245.0.0/16"]
}]Both switches default to true, including when explicitly passed null.
VPC IPv6 defaults remain enabled. Disabling an IPv4 default requires at least
one explicit rule for that load balancer; an empty dynamic ingress block does
not reliably remove existing rules in the underlying security group resource.
For existing clusters, add and apply the explicit rules first, then disable the IPv4 defaults in a separate apply. Review plans for SG ingress changes only, without replacing the LB or SG. Test every required client path after each apply. For PrivateLink with inbound SG enforcement enabled, allow the actual client IP ranges, including any required clients outside the endpoint VPC; allowing only the provider VPC or endpoint interface IPs is insufficient.
Note
Starting with version 7.6.0, specifying the AWS region via the region input variable is required.
Users must provide a valid region string (lowercase letters and digits, optionally separated by single hyphens), for example us-east-1.
General steps to update EKS cluster:
-
check AWS documentation about EKS cluster update procedure
-
update information about addons versions in repository teraform-aws-eks and release new module version
locals {
# https://docs.aws.amazon.com/eks/latest/userguide/pod-id-agent-setup.html
# aws eks describe-addon-versions --addon-name eks-pod-identity-agent | jq '.addons[0].addonVersions[0]'
eks_pod_identity_agent_version = {
"1.29" = "v1.3.5-eksbuild.2"
"1.30" = "v1.3.5-eksbuild.2"
"1.31" = "v1.3.5-eksbuild.2"
"1.32" = "v1.3.5-eksbuild.2"
}
}- upgrade addons for EKS clusters, by bump version of module teraform-aws-eks to the latest release
diff --git a/internal-tools/dev/us-east-1/eks.tf b/internal-tools/dev/us-east-1/eks.tf
index a95261645..9cf8d04b9 100644
--- a/internal-tools/dev/us-east-1/eks.tf
+++ b/internal-tools/dev/us-east-1/eks.tf
@@ -14,7 +14,7 @@ module "acm" {
}
module "eks" {
- source = "git@github.com:worldcoin/terraform-aws-eks?ref=v4.4.2"
+ source = "git@github.com:worldcoin/terraform-aws-eks?ref=v4.5.0"
cluster_name = format("tools-%s-%s", var.environment, data.aws_region.current.region)
cluster_version = "1.32"
- upgrade EKS control plane, by bump version in variable cluster_version for each cluster
--- a/internal-tools/dev/us-east-1/eks.tf
+++ b/internal-tools/dev/us-east-1/eks.tf
@@ -17,7 +17,7 @@ module "eks" {
source = "git@github.com:worldcoin/terraform-aws-eks?ref=v4.5.0"
cluster_name = format("tools-%s-%s", var.environment, data.aws_region.current.region)
- cluster_version = "1.31"
+ cluster_version = "1.32"
environment = var.environment
vpc_config = module.vpc.config
extra_role_mapping = module.sso_roles.default_mappings
Note
The control plane can only be updated by +1 version!!!
Note
Repeat this step many times to get right EKS cluster version
- observe node group rotation, after upgrade control-plane/launch-template/AMIs not always it is done automatically. From time to time manual operation is required here to kill pods with pdb/annotations.
Note
Kubelet compatibility is +3 versions, and node group rotation is not always required and can be done once in the end
Note
Please be carefull with EKS crypto nodes rotation, and don't do this without information on slack channel #planned-outages
Alternative approche is to leave them with status Read/SchedulingDisabled and wait when they rotate with apps deployment
- schedule
start-instance-refreshfor node group used to keep infrastructure pods
aws autoscaling start-instance-refresh --auto-scaling-group-name eks-node-tools-dev-us-east-1 --region us-east-1 --profile wld-internal-tools-dev --output json- observe
describe-instance-refreshesfor node group used to keep infrastructure pods
aws autoscaling describe-instance-refreshes --auto-scaling-group-name eks-node-tools-dev-us-east-1 --region us-east-1 --profile wld-internal-tools-dev --output jsonManual upgrade is required with below command, and terraform apply after execution.
aws eks update-cluster-version --region ... --name ... --kubernetes-version 1.29Works like a charm without of any manual operation. Just plan/apply workspace with TFE.
To remove the cluster you have to:
-
Strip finalizers from Gateway API Gateways:
# Gateway API Gateways live in kube-system and carry finalizers that block CRD/LB deletion. # Check which exist first: kubectl get gateway -A. Not all four may be present. kubectl -n kube-system delete gateway gw-ext-alb gw-ext-nlb gw-int-alb gw-int-nlb --wait=false 2>/dev/null || true kubectl -n kube-system patch gateway gw-ext-alb -p '{"metadata":{"finalizers":null}}' --type=merge 2>/dev/null || true kubectl -n kube-system patch gateway gw-ext-nlb -p '{"metadata":{"finalizers":null}}' --type=merge 2>/dev/null || true kubectl -n kube-system patch gateway gw-int-alb -p '{"metadata":{"finalizers":null}}' --type=merge 2>/dev/null || true kubectl -n kube-system patch gateway gw-int-nlb -p '{"metadata":{"finalizers":null}}' --type=merge 2>/dev/null || true
-
Set these flags, the module will remove every usage of the Kubernetes provider and allow you to remove the cluster module without any errors. Setting
enable_deletion_protection = falsedisables deletion protection on the Gateway API NLB/ALB load balancers so they can be removed by Terraform.efs_csi_driver_enabled = false kubernetes_provider_enabled = false enable_deletion_protection = false
-
If above PR
applyfails (possible reasons: race condition -aws_authremoved too soon, or Gateway API CRDs/Gateways timing out due to remaining finalizers), remove the affectedkubernetes_*resources from state and rerunapply:terraform state list |grep kubernetes_ terraform state rm ... -
Remove module invocation to finally delete cluster itself.
-
If above PR
applyfails on deleting autoscaling groups or security groups, check for leftover EC2 instances (terminate them and rerunapply) or other security groups referencing the cluster SGs via ingress/egress rules (remove the referencing rules and rerunapply).
See also:
docs/EKS_CLUSTER_REMOVAL.mdinworldcoin/infrastructurefor a more detailed runbook including pre-steps, failure mode troubleshooting, and workspace-specific guidance.
| Name | Version |
|---|---|
| terraform | >= 1.11.0 |
| aws | >= 6.57.1 |
| cloudflare | >= 5.8 |
| datadog | >= 3.0 |
| kubernetes | >= 2.0 |
| random | >= 3.3 |
| time | >= 0.9 |
| tls | >= 4.0 |
| Name | Version |
|---|---|
| aws | >= 6.57.1 |
| cloudflare | >= 5.8 |
| datadog | >= 3.0 |
| kubernetes | >= 2.0 |
| random | >= 3.3 |
| terraform | n/a |
| time | >= 0.9 |
| tls | >= 4.0 |
| Name | Description | Type | Default | Required |
|---|---|---|---|---|
| access_entries | Map of access entries to add to the cluster | map(object({ |
{} |
no |
| additional_cluster_security_group_rules | Additional cluster security group rules | list(object({ |
[] |
no |
| additional_open_ports | Additional ports accessible from the Internet for the ALB | set(object({ |
[] |
no |
| additional_security_group_rules | Additional security group rules | list(object({ |
[] |
no |
| alb_idle_timeout | The time in seconds that the connection is allowed to be idle | number |
60 |
no |
| alb_logs_bucket_id | The ID of the S3 bucket to store logs in for ALB. | string |
n/a | yes |
| argocd_image_updater_enabled | Whether to create a role and Pod Identity association for the argocd/argocd-image-updater Deployment so it can read ECR image tags without IMDS | bool |
true |
no |
| argocd_role_arn | The ARN of the remote ArgoCD role used to assume eks-cluster role | string |
null |
no |
| authentication_mode | The authentication mode for the cluster. Valid values are CONFIG_MAP, API or API_AND_CONFIG_MAP |
string |
"API_AND_CONFIG_MAP" |
no |
| aws_autoscaling_group_enabled | Whether to enable AWS Autoscaling group | bool |
true |
no |
| aws_load_balancer_iam_role_enabled | Whether to enable the IAM role for the AWS Load Balancer | bool |
true |
no |
| cluster_endpoint_public_access | Indicates whether or not the Amazon EKS public API server endpoint is enabled | bool |
false |
no |
| cluster_name | The name of the cluster. Has to be unique per region per account. | string |
n/a | yes |
| cluster_version | The Kubernetes version to use for the cluster. | string |
"1.35" |
no |
| coredns_max_replicas | Maximum number of replicas for CoreDNS | number |
10 |
no |
| coredns_min_replicas | Minimum number of replicas for CoreDNS | number |
2 |
no |
| datadog_api_key | Datadog API key. Stored in kube-system namespace as a secret. | string |
n/a | yes |
| deploy_desired_vs_status_critical | Threshold for critical for Desired pods vs current pods (Deployments) | number |
10 |
no |
| deploy_desired_vs_status_evaluation_period | Evaluation period for Desired pods vs current pods (Deployments) | string |
"last_15m" |
no |
| deploy_desired_vs_status_warning | Threshold for warning for Desired pods vs current pods (Deployments) | number |
1 |
no |
| dockerhub_pull_through_cache_repositories_arn | The ARN of the repositories to allow the EKS node group to pull images from the DockerHub pull-through cache. | string |
"arn:aws:ecr:us-east-1:507152310572:repository/docker-cache/*" |
no |
| drop_invalid_header_fields | Drop invalid header fields | bool |
true |
no |
| ebs_csi_metadata_sources | Comma-separated metadata sources for the aws-ebs-csi-driver node plugin, rendered as node.metadataSources. Defaults to kubernetes because every cluster runs at IMDS hop limit 1, where the driver's own default order (imds then kubernetes) always fails the imds attempt first and costs a 5s timeout plus one error log per node start (INFRA-7097). Set to null to send no configuration and let the driver use its own default, which is only meaningful on a cluster at hop limit 2 or higher. Valid tokens: imds, kubernetes, metadata-labeler. | string |
"kubernetes" |
no |
| ecr_credentials_sync_enabled | Whether to create a role and Pod Identity association for the argocd/ecr-credentials-sync CronJob so it can call ecr:GetAuthorizationToken without IMDS | bool |
true |
no |
| efs_csi_driver_enabled | Whether to enable the EFS CSI driver (IAM Role & StorageClass). | bool |
false |
no |
| efs_provisioned_throughput_in_mibps | Provisioned EFS throughput in MiB/s; required only in provisioned mode. | number |
null |
no |
| efs_throughput_mode | Throughput mode for the EKS EFS filesystem. | string |
"bursting" |
no |
| eks_node_group | Configuration for EKS node group | object({ |
null |
no |
| enable_aws_load_balancer_controller_explicit_deny | Safety switch: set to false to disable creating the aws-load-balancer-controller explicit deny policy and attachment. | bool |
true |
no |
| enable_deletion_protection | Whether to enable deletion protection on the Gateway API NLB/ALB load balancers. Set to false before destroying the cluster. | bool |
true |
no |
| enclave_tracks | Additional enclave tracks for multi-version deployments. Key is used as track identifier. | map(object({ |
{} |
no |
| enclaves_cpu_allocation | Number of CPUs to allocate for Nitro Enclaves per node | string |
"4" |
no |
| enclaves_instance_type | Instance type for Nitro Enclaves | string |
"m7a.2xlarge" |
no |
| enclaves_memory_allocation | Memory in MiB to allocate for Nitro Enclaves per node | string |
"4096" |
no |
| environment | Environment of cluster | string |
n/a | yes |
| external_cert_arn | ACM certificate ARN for external load balancers. | string |
null |
no |
| external_check_locations | List of DD locations to check cluster availability from | list(string) |
[ |
no |
| external_tls_listener_version | The version of the TLS listener to use for external ALB. | string |
"1.3" |
no |
| extra_role_mapping | Extra role mappings to add to the aws-auth configmap. | list(object({ |
[] |
no |
| gateway_api_crds_enabled | Install Gateway API and AWS LBC CRDs. Must be enabled before gateway_api_external_enabled or gateway_api_internal_enabled. | bool |
false |
no |
| gateway_api_ext_alb_extra_certificates | Additional ACM certificate ARNs to attach as SNI certificates on the external ALB HTTPS listener, alongside the default certificate. Lets callers add SNI certs without restating listener defaults. Ignored when gateway_api_ext_alb_listener_configs is set (that override wins). | list(string) |
[] |
no |
| gateway_api_ext_alb_listener_configs | Override LoadBalancerConfiguration listenerConfigurations for the external ALB. When null, defaults to HTTPS:443 with external cert and SSL policy. | any |
null |
no |
| gateway_api_ext_alb_listeners | Gateway listeners for the external ALB (gw-ext-alb). | any |
[ |
no |
| gateway_api_ext_nlb_listener_configs | Override LoadBalancerConfiguration listenerConfigurations for the external NLB. When null, defaults to TLS:443 with external cert and SSL policy. | any |
null |
no |
| gateway_api_ext_nlb_listeners | Gateway listeners for the external NLB (gw-ext-nlb). | any |
[ |
no |
| gateway_api_external_alb_sg_rules | Additional ingress rules for the external Gateway API ALB's backend security group (terraform-aws-alb's backend_ingress_rules — the frontend SG is managed separately via Cloudflare IPs/open_to_all), appended to the module's own defaults — this does not replace them. | list(object({ |
[] |
no |
| gateway_api_external_enabled | Create internet-facing ALB and NLB for Gateway API (external, external-nonhttp). Requires gateway_api_crds_enabled = true. | bool |
false |
no |
| gateway_api_external_nlb_sg_rules | Additional LB security group ingress rules for the external Gateway API NLB, appended to the module's own defaults (ports 80/443 from Cloudflare IPs) — this does not replace them. | list(object({ |
[] |
no |
| gateway_api_int_alb_extra_certificates | Additional ACM certificate ARNs to attach as SNI certificates on the internal ALB HTTPS listener, alongside the default certificate. Lets callers add SNI certs without restating listener defaults. Ignored when gateway_api_int_alb_listener_configs is set (that override wins). | list(string) |
[] |
no |
| gateway_api_int_alb_listener_configs | Override LoadBalancerConfiguration listenerConfigurations for the internal ALB. When null, defaults to HTTPS:443 with internal cert and SSL policy. | any |
null |
no |
| gateway_api_int_alb_listeners | Gateway listeners for the internal ALB (gw-int-alb). | any |
[ |
no |
| gateway_api_int_nlb_listener_configs | Override LoadBalancerConfiguration listenerConfigurations for the internal NLB. When null, defaults to TLS:443 with internal cert and SSL policy. | any |
null |
no |
| gateway_api_int_nlb_listeners | Gateway listeners for the internal NLB (gw-int-nlb). | any |
[ |
no |
| gateway_api_internal_alb_default_ipv4_ingress_enabled | Allow HTTPS from 10.0.0.0/8 to the internal Gateway API ALB. Set false with explicit gateway_api_internal_alb_sg_rules to restrict IPv4 clients. VPC IPv6 defaults are preserved. | bool |
true |
no |
| gateway_api_internal_alb_sg_rules | Ingress rules appended to the internal Gateway API ALB defaults. Set gateway_api_internal_alb_default_ipv4_ingress_enabled = false to remove the default 10.0.0.0/8 rule. VPC IPv6 defaults are preserved. | list(object({ |
[] |
no |
| gateway_api_internal_enabled | Create internal ALB and NLB for Gateway API (internal, internal-nonhttp). Requires gateway_api_crds_enabled = true. | bool |
false |
no |
| gateway_api_internal_nlb_default_ipv4_ingress_enabled | Allow HTTP/HTTPS from 10.0.0.0/8 to the internal Gateway API NLB. Set false with explicit gateway_api_internal_nlb_sg_rules to restrict IPv4 clients. VPC IPv6 defaults are preserved. | bool |
true |
no |
| gateway_api_internal_nlb_sg_rules | Ingress rules appended to the internal Gateway API NLB defaults. Set gateway_api_internal_nlb_default_ipv4_ingress_enabled = false to remove the default 10.0.0.0/8 rules. VPC IPv6 defaults are preserved. | list(object({ |
[] |
no |
| gateway_api_lb_name_prefix | Prefix for Gateway API load balancer names. Defaults to cluster_name. Override when cluster_name is too long to fit within the 32-char AWS LB name limit (prefix + suffix like '-gw-ext-alb' must be <= 32). | string |
null |
no |
| gha_cidr_eu_central_1 | GitHub Actions CIDR block for eu-central-1 | string |
"10.52.0.0/20" |
no |
| gha_cidr_us_east_1 | GitHub Actions CIDR block for us-east-1 | string |
"10.0.96.0/20" |
no |
| http_put_response_hop_limit | IP TTL applied to IMDSv2 token responses, set as metadata_options.http_put_response_hop_limit on every node launch template. Must be between 1 and 64. At 1 the response stays in the host network namespace, so a Pod one hop away cannot reach 169.254.169.254 and assume the node IAM role. Pods with hostNetwork: true share that namespace and are unaffected. | number |
1 |
no |
| internal_cert_arn | ACM certificate ARN for internal load balancers (falls back to external_cert_arn). If empty, internal_nlb_acm_arn is used for backwards compatibility. | string |
"" |
no |
| internal_nlb_acm_arn | (Deprecated: use internal_cert_arn) The ARN of the certificate to use for internal NLB. | string |
"" |
no |
| internal_tls_listener_version | The version of the TLS listener to use for internal NLB. | string |
"1.3" |
no |
| kube_ops_enabled | Whether to create a role and association for kube-ops | bool |
true |
no |
| kubelet_extra_args | kubelet extra args to pass to the node group | string |
"--register-with-taints=critical:NoExecute" |
no |
| kubernetes_provider_enabled | Create Kubernetes resources. Set to false for initial cluster creation and before cluster removal. | bool |
true |
no |
| memory_limits_low_perc_enabled | Enable memory limits low percentage alert | bool |
false |
no |
| monitor_system_workload_only | Monitor system workloads only (infra team). | bool |
false |
no |
| monitoring_enabled | Whether to enable monitoring (Datadog). | bool |
true |
no |
| monitoring_external_enabled | Whether to enable external monitoring (Datadog Synthetics). | bool |
true |
no |
| monitoring_notification_channel | The Datadog notification channel to use for monitoring alerts (infra team). | string |
"@slack-TFH-infrastructure-alerts" |
no |
| monitoring_reachability_fail_locations | Number of locations to fail to trigger the reachability test | number |
2 |
no |
| monitoring_reachability_failure_duration | Time after first error when the reachability test is triggered | number |
300 |
no |
| monitoring_user_workload_excluded_namespaces | Namespaces excluded from user Kubernetes workload monitors. Cluster-wide storage and the separate OOM monitor are unchanged. | set(string) |
[] |
no |
| monitoring_user_workload_notification_channel | Monitor user workload and send alerts to team channel/webhook). | string |
"" |
no |
| monitoring_user_workload_team | The team tag for monitoring user workload. | string |
"" |
no |
| mtls_enabled | Enable mutual TLS (mTLS) on the ALB TLS listener | bool |
true |
no |
| nlb_az_affinity | Per-NLB AZ-affinity overrides for the gateway-api NLBs (gateway_api_internal, gateway_api_external). Unset sub-fields preserve prior module behavior (enable_cross_zone_load_balancing = true, dns_record_client_routing_policy = "any_availability_zone"). | object({ |
{} |
no |
| node_instance_profile_inline_policies | Inline policies to attach to the node instance profile. Key is the name of the policy, value is the policy document. | map(string) |
{} |
no |
| node_monitoring_agent_enabled | Enable node monitoring agent | bool |
false |
no |
| on_demand_base_capacity | The number of minimum on-demand instances to launch. | number |
1 |
no |
| on_demand_percentage_above_base_capacity | The percentage of on-demand instances to use above the base capacity | number |
50 |
no |
| open_to_all | Set to true if you want to open the cluster to all traffic from internet |
bool |
false |
no |
| public_access_cidrs | List of CIDR blocks. Indicates which CIDR blocks can access the Amazon EKS public API server endpoint when enabled. | list(string) |
[ |
no |
| region | AWS Region | string |
n/a | yes |
| replicaset_incomplete_enabled | Whether to enable the ReplicaSet Incomplete alert. | bool |
true |
no |
| replicaset_unavailable_enabled | Whether to enable the ReplicaSet Unavailable alert. | bool |
true |
no |
| s3_mountpoint_csi_driver_enabled | Whether to enable the S3 mountpoint CSI driver | bool |
false |
no |
| s3_mountpoint_csi_s3_bucket_arns | List of S3 bucket ARNs to allow access from the S3 mountpoint CSI driver | list(string) |
[ |
no |
| static_autoscaling_group | Configuration for static autoscaling group | object({ |
null |
no |
| storage_class | Configuration for the storage class that defines how volumes are allocated in Kubernetes. | object({ |
{ |
no |
| tfe_cidr | Terraform Enterprise CIDR block | string |
"10.52.160.0/20" |
no |
| use_private_subnets | Set to true to place EKS node groups in private subnets, false to use public subnets. |
bool |
true |
no |
| use_private_subnets_for_internal_nlb | Set to true if you want to use private subnets for internal NLB |
bool |
false |
no |
| vector_audit_enabled | Create per-cluster EKS audit-log pipeline (Firehose + CW Logs subscription filter → central Vector aggregator). Requires vector_audit_firehose_access_key. | bool |
false |
no |
| vector_audit_endpoint_url | HTTPS URL of the central Vector aggregator's aws_kinesis_firehose source. | string |
"https://vector.worldcoin.dev/" |
no |
| vector_audit_firehose_access_key | Shared access key sent by Firehose in X-Amz-Firehose-Access-Key; validated by Vector's aws_kinesis_firehose source. | string |
null |
no |
| vector_audit_org | Organisation identifier written into the S3 key prefix (kubernetes-audit/org=/...). | string |
"tfh" |
no |
| vector_audit_s3_backup_bucket | S3 bucket name for Firehose FailedDataOnly backup (failed deliveries to Vector). | string |
"wld-log-archive" |
no |
| vector_s3_bucket_arns | S3 bucket ARNs Vector is allowed to write logs to. Empty disables Vector IAM role and pod identity association. | list(string) |
[ |
no |
| vpc_cni_enable_network_policy | Enable Kubernetes NetworkPolicy enforcement via the VPC CNI node agent | bool |
false |
no |
| vpc_cni_enable_pod_eni | Enable pod ENI support | bool |
false |
no |
| vpc_cni_enable_prefix_delegation | Enable prefix delegation for IPv6, allocate IPs in /28 blocks (instead of all at once) | bool |
false |
no |
| vpc_cni_external_snat | Needed to enable cross-vpc pod-to-pod communication - see: https://github.com/aws/amazon-vpc-cni-k8s?tab=readme-ov-file#aws_vpc_k8s_cni_externalsnat | string |
false |
no |
| vpc_cni_pod_security_group_enforcing_mode | Set pod security group enforcing mode | string |
"standard" |
no |
| vpc_cni_version_override | The version of the VPC CNI plugin to use. If not specified, the default version for the cluster version will be used. | string |
"" |
no |
| vpc_cni_warm_eni_target | Number of ENIs to keep warm for each node to speed up pod scheduling | string |
"1" |
no |
| vpc_cni_warm_ip_target | Number of IPs to keep warm for each node to speed up pod scheduling | string |
"4" |
no |
| vpc_config | VPC configuration from aws/vps module | object({ |
n/a | yes |
| Name | Description |
|---|---|
| cluster_certificate_authority_data | Base64 encoded certificate data required to communicate with the cluster |
| cluster_endpoint | Endpoint for your Kubernetes API server |
| cluster_oidc_issuer_url | The OIDC issuer URL for the EKS cluster |
| gateway_api_external_alb_arn | ARN of the external Gateway API ALB |
| gateway_api_external_alb_dns_name | DNS name of the external Gateway API ALB |
| gateway_api_external_nlb_arn | ARN of the external Gateway API NLB |
| gateway_api_external_nlb_dns_name | DNS name of the external Gateway API NLB |
| gateway_api_internal_alb_arn | ARN of the internal Gateway API ALB |
| gateway_api_internal_alb_dns_name | DNS name of the internal Gateway API ALB |
| gateway_api_internal_nlb_arn | ARN of the internal Gateway API NLB |
| gateway_api_internal_nlb_dns_name | DNS name of the internal Gateway API NLB |
| name | The name of the cluster |
| node_security_group_id | The security group ID of the EKS nodes |