apigee module does not map correctly the organization properties

[nc-ivan-lecreurer]

We are using the v35.0.0 of the apigee module. We have created the organization without properties.

module "apigee" {
  source     = "github.com/terraform-google-modules/cloud-foundation-fabric//modules/apigee?ref=v35.0.0"
  project_id = var.project_id
  organization = {
    display_name            = var.org_display_name
    description             = var.org_description
    runtime_type            = "CLOUD"
    billing_type            = var.billing_type
    database_encryption_key = module.kms-org-db.key_ids["org-db"]
    analytics_region        = var.ax_region
    disable_vpc_peering     = var.disable_vpc_peering
    api_consumer_data_location = "europe-west4"
  }
  envgroups    = local.envgroups
  environments = var.apigee_environments
  instances    = local.instances
}

On trying to run again Terraform tries to recreate the entire organization because it has two new properties namely:

• "features.hybrid.enabled"
• "features.mart.connect.enabled"

I've tried to set them to match as follows:

module "apigee" {
  source     = "github.com/terraform-google-modules/cloud-foundation-fabric//modules/apigee?ref=v35.0.0"
  project_id = var.project_id
  organization = {
    display_name            = var.org_display_name
    description             = var.org_description
    runtime_type            = "CLOUD"
    billing_type            = var.billing_type
    database_encryption_key = module.kms-org-db.key_ids["org-db"]
    analytics_region        = var.ax_region
    disable_vpc_peering     = var.disable_vpc_peering
    api_consumer_data_location = "europe-west4"
    properties = {
       "features.hybrid.enabled" = "true"
       "features.mart.connect.enabled" = "true"
    }
  }
  envgroups    = local.envgroups
  environments = var.apigee_environments
  instances    = local.instances
}

But even that isn't matched. See part of the trace below.

  # module.apigee-x-core.module.apigee.google_apigee_organization.organization[0] must be replaced
-/+ resource "google_apigee_organization" "organization" {
      + api_consumer_data_location           = "europe-west4" # forces replacement
      ~ apigee_project_id                    = "la939134d21e1f95cp-tp" -> (known after apply)
      ~ ca_certificate                       = "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURQekNDQWllZ0F3SUJBZ0lRYTNJT1dMZXlHRzFUZmlzeXVVU3ZWVEFOQmdrcWhraUc5dzBCQVFzRkFEQXYKTVMwd0t3WURWUVFERXlRNVptUm1NREZsTUMxalkyRXpMVFJsTkdFdFlUYzJOUzFsTVRBeU1HRmpNbVk0TVRBdwpIaGNOTWpReE1USXhNRGN6TURJeldoY05NelF4TVRFNU1EY3pNREl6V2pBdk1TMHdLd1lEVlFRREV5UTVabVJtCk1ERmxNQzFqWTJFekxUUmxOR0V0WVRjMk5TMWxNVEF5TUdGak1tWTRNVEF3Z2dFaU1BMEdDU3FHU0liM0RRRUIKQVFVQUE0SUJEd0F3Z2dFS0FvSUJBUURWTGZjU3MxVStSOWZ4QTROYzFManRuWUtiKzAxZGUzS2ttc1NvNlRtUwpUQXNBbVpsd0lwSExiMzFsa2ZYbWpBdUd3T0FxNmlxVmNqK0xYRjF4TjRWRENkQ0dRaWNNZDcrZzBzWkJURXBmCi9Rb2g1WjRibXpFS3NIZEp1WDNnK2hFa2hmTG9OSmVEalNpOHBsOVN3OWZUcjVJVlV1ZStPMENrNm1NRU0rU0UKdmFDRGJaMjdvemh6MHBZZEs5S25JeFAzeXJmYm40emZ6cDhDaWFZK3pxNVBrOTBzTkkzRUlrTE51YkY5YzhPcwp1eWJwaHJhc2QzUEpqc2YzaU9xWXRpZW0xNk1remNWNlJMajRCMUhPSnhmcng1Rm4zVEtUKzZ0US9jeDBsWllPCldURzBPaVRIQ09DUkl2VjJFaWhTWUpjczEwc3d1UzZrZ2VTemhTM0ZhQ2szQWdNQkFBR2pWekJWTUE0R0ExVWQKRHdFQi93UUVBd0lDcERBVEJnTlZIU1VFRERBS0JnZ3JCZ0VGQlFjREFUQVBCZ05WSFJNQkFmOEVCVEFEQVFILwpNQjBHQTFVZERnUVdCQlFRU2E4dHF4SkVQVGlTUVl6eTFUSkFRbldMeXpBTkJna3Foa2lHOXcwQkFRc0ZBQU9DCkFRRUFGYjZZcjlSRVdkMTE4ZW0vOTFmY0ZhejNzREk4QzZqRVh4LzZ3eGZzZXBWaGFObEJrYnlLUjJheDdYcmIKdjUwZWkvTE1oREkzMFJpWnR5b096NnRPVWEyU1Y5UHN6ZWVnU1JSajJuWkpiRjB3d1FaT1d5WisvaU9hV25hbgpOM3Q2UjV4d1d2Wk94Y2tEamFzc0NYWkVkbWRYTDJISWtCL1RkenRWYjhsK05tL1p2QXd1SjdZQUlrdUNHREQ1CklGU2R3cElEeW41akNVaVJISENEZGhxbnd3NDlsd2RVb0VNbGNqem02ZHZwYmFtb2hJNWMvaFU2UXMxU0lDRksKMGxycmhvSm83SjJ0a0N2Q3V5dzl0MHVXZ1BFb2MyMllXblMybVdDc0tSNk5KS1R0bnRsbHBFNnpvUGpsbUYragpCN2NsZEZhL0FGTXNJdEUxRlNFMkpGdEtKdz09Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K" -> (known after apply)
      ~ id                                   = "organizations/lhg-eu-w0p-mmg-api01" -> (known after apply)
      ~ name                                 = "lhg-eu-w0p-mmg-api01" -> (known after apply)
      ~ subscription_type                    = "PAID" -> (known after apply)
        # (7 unchanged attributes hidden)

      ~ properties {
          ~ property {
              ~ name  = "features.hybrid.enabled" -> "0"
              - value = "true" -> null
            }
          ~ property {
              ~ name  = "features.mart.connect.enabled" -> "0"
              - value = "true" -> null
            }
        }
    }

So I see two problems, some default properties are created and we have no way to setting the properties to match afterwards.

[danistrebel]

Yes, I think the main problem is with the new api_consumer_data_location attribute.

That wasn't part of the API initially so it forces a replacement of the whole org. You could try to re-import state but the underlying problem seems to be with the breaking change in the upstream fabric module not with the example here.

[nc-ivan-lecreurer]

The management API gives us the following response (I've removed the non-relevant parts):

{
    ...
    "environments": [
        "prod"
    ],
    "properties": {
        "property": [
            {
                "name": "features.mart.connect.enabled",
                "value": "true"
            },
            {
                "name": "features.hybrid.enabled",
                "value": "true"
            }
        ]
    },
    "analyticsRegion": "europe-west4",
    ...
}

Does this mean that api_consumer_data_location is used to create the org but then it is not compared correctly afterwards hence Terraform thinks I'm adding that attribute?

[nc-ivan-lecreurer]

I have commented the attribute and now the only reason to update the org is this difference:

  # module.apigee-x-core.module.apigee.google_apigee_organization.organization[0] will be updated in-place
  ~ resource "google_apigee_organization" "organization" {
        id                                   = "organizations/lhg-eu-w0p-mmg-api01"
        name                                 = "lhg-eu-w0p-mmg-api01"
        # (10 unchanged attributes hidden)

      ~ properties {
          ~ property {
              ~ name  = "features.hybrid.enabled" -> "0"
              - value = "true" -> null
            }
          ~ property {
              ~ name  = "features.mart.connect.enabled" -> "0"
              - value = "true" -> null
            }
        }
    }