Skip to content

Deploy Module CodeDeploy Race Condition #700

@Levistator

Description

@Levistator

Description

When I am attempting to use the Deploy module with almost no customization, the Code Deploy script fails with a lack of permissions error. If I immediately afterwards go into the AWS Console and retry that Deployment then it works.
This leads me to expect that there is a race condition happening.

  • ✋ I have searched the open/closed issues and my issue is not listed.

⚠️ Note

Before you submit an issue, please perform the following first:

  1. Remove the local .terraform directory (! ONLY if state is stored remotely, which hopefully you are following that best practice!): rm -rf .terraform/
  2. Re-initialize the project root to pull down modules: terraform init
  3. Re-attempt your terraform plan or apply and check if the issue still persists

Versions

  • Module version [Required]: 7.21.0

  • Terraform version: 1.12.2

  • Provider version(s):
    • provider registry.terraform.io/hashicorp/aws v5.100.0
    • provider registry.terraform.io/hashicorp/external v2.3.5
    • provider registry.terraform.io/hashicorp/local v2.5.3
    • provider registry.terraform.io/hashicorp/null v3.2.4
    • provider registry.terraform.io/hashicorp/random v3.7.2
    • provider registry.terraform.io/kreuzwerker/docker v3.6.2

Reproduction Code [Required]

module "orchestrator_deploy" {
  source  = "terraform-aws-modules/lambda/aws//modules/deploy"
  version = "7.21.1"

  depends_on = [
    module.orchestrator,
    module.orchestrator_alias
  ]

  description = "${var.service_name} deployment!"

  alias_name     = module.orchestrator_alias.lambda_alias_name
  function_name  = module.orchestrator.lambda_function_name
  target_version = module.orchestrator.lambda_function_version

  create_app = true
  app_name   = var.service_name

  create_deployment_group = true
  deployment_group_name   = "${var.service_name}-deployment-group"

  create_deployment          = true
  run_deployment             = true
  deployment_config_name     = "CodeDeployDefault.LambdaAllAtOnce"
  save_deploy_script         = false
  wait_deployment_completion = true
  force_deploy               = false
}

Steps to reproduce the behavior:

  1. Attempt to use the module with a Docker-based Lambda

Expected behavior

The Code Deploy Script works just fine and doesn't fail due to a permissions error.

Actual behavior

When I am attempting to deploy a code change to my lambda, I see that the AWSCodeDeployRoleForLambda policy attachment is getting destroyed and recreated every single time. With this the Deploy Script exec runs after the policy attachment is destroyed, but before the attachment is completed.

Then the deployment script returns that it failed due to the following error.

The IAM role arn:aws:iam::{Account}:role/{service_name}-codedeploy does not give you permission to perform operations in the following AWS service: AWSLambda. Contact your AWS administrator if you need help. If you are an AWS administrator, you can grant permissions to your users or groups by creating IAM policies.

Terminal Output Screenshot(s)

Plan Output

  # module.orchestrator_deploy.data.aws_iam_policy_document.assume_role[0] will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_iam_policy_document" "assume_role" {
      + id            = (known after apply)
      + json          = (known after apply)
      + minified_json = (known after apply)

      + statement {
          + actions = [
              + "sts:AssumeRole",
            ]
          + effect  = "Allow"

          + principals {
              + identifiers = [
                  + "codedeploy.amazonaws.com",
                ]
              + type        = "Service"
            }
        }
    }

  # module.orchestrator_deploy.aws_iam_role.codedeploy[0] will be updated in-place
  ~ resource "aws_iam_role" "codedeploy" {
      ~ assume_role_policy    = jsonencode(
            {
              - Statement = [
                  - {
                      - Action    = "sts:AssumeRole"
                      - Effect    = "Allow"
                      - Principal = {
                          - Service = "codedeploy.amazonaws.com"
                        }
                    },
                ]
              - Version   = "2012-10-17"
            }
        ) -> (known after apply)
        id                    = "avm-github-runners-codedeploy"
        name                  = "avm-github-runners-codedeploy"
        tags                  = {}
        # (11 unchanged attributes hidden)
    }

  # module.orchestrator_deploy.aws_iam_role_policy_attachment.codedeploy[0] must be replaced
-/+ resource "aws_iam_role_policy_attachment" "codedeploy" {
      ~ id         = "avm-github-runners-codedeploy-20250904200636772200000001" -> (known after apply)
      ~ policy_arn = "arn:aws:iam::aws:policy/service-role/AWSCodeDeployRoleForLambda" -> (known after apply) # forces replacement
        # (1 unchanged attribute hidden)
    }

Apply Ordering

  1. null_resource.deploy[0]: Destroying... [id=3803226194750982394]
  2. aws_iam_role_policy_attachment.codedeploy[0]: Destroying...
  3. aws_iam_role_policy_attachment.codedeploy[0]: Creating...
  4. null_resource.deploy[0]: Creating...
  5. null_resource.deploy[0]: Provisioning with 'local-exec'...
  6. aws_iam_role_policy_attachment.codedeploy[0]: Creation complete after 0s
  7. null_resource.deploy[0] (local-exec): Deployment failed!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions