central place to define module versions

[chris-oconnor-plurilock]

Terraform Version

Terraform v1.2.3
on darwin_amd64
+ provider registry.terraform.io/hashicorp/aws v4.12.1
+ provider registry.terraform.io/hashicorp/random v3.3.2

Your version of Terraform is out of date! The latest version
is 1.2.8. You can update by downloading from https://www.terraform.io/downloads.html

Use Cases

in large terraform files its annoying to have to search replace module versions and source (if switching from local to remote during development). This is especially worse if you are including the same module multiple times

Attempted Solutions

it's possible to do something to help with multi calls to the same module by creating a wrapper

/main.tf

module alpha{
  source = "./wrappers/a"
  something_useful = "cake"
}
module beta{
  source = "./wrappers/a"
  something_useful = "or death"
}

/wrappers/a/main.tf

variable something_useful{}

module wrapped{
   source = "app.terraform.io/somegreatmodule"
   version = "0.0.1"
   something_useful = var.something_useful
}

Then you only have to update the relatively small files inside /wrappers/*/main.tf but this is annoying as you have to duplicate in the variables inside the wrapper module and pass them through and potential outputs

Proposal

Provide a central place to define module version and source then inside tf files refer to this external constant so that a single location can be defined making version bumping easy and less error prone

Something if like below would seem the terraformy way

packages.tf

module_def a {
   source = "app.terraform.io/somethingamazing"
   version = "1.1.0"
}

module_def b {
   source = "../somethinglocal"
}

main.tf

module cake {
   source = module_def.a
   module_args....

}

module tea {
   source = module_def.a   
  module_args....
}
module biscuits {
   source = module_def.b
   module_args....
}

References

• module source dynamically set #25587
• terraform get: can't use variable in module source parameter? #1439

[marianod92]

Hi @chris-oconnor-plurilock

I have the same need to be able to parameterize the versions of the modules between environments and also, it could apply for the complete source=.

I am following other associated issues but at the moment, there is no native Terraform functionality that allows it.

Thank you for creating the issue and giving visibility to this problem.

[kmoe]

Thanks for the feature request. I see the problem here and there are several ways we could solve it.

[apparentlymart]

Indeed thanks for starting this discussion!

This reminds me of a similar situation that historically existed for providers, where we had overloaded the version argument in configuration to represent two distinct pieces of information:

1. Which set of versions of this provider is this module compatible with?
2. Which specific version of this provider are we currently using? (Which must always be a member of the intersection of all of the sets of versions all modules are compatible with.)

We resolved that ambiguity/conflict in Terraform v0.13 by introducing the dependency lock file as a new place to capture the second piece of information, and thereby left the module itself to only worry about the first piece of information.

The situation isn't quite the same for modules as it is for providers, because Terraform allows each individual module call to specify an entirely-separate version of a module, and allowing that is a hard requirement to allow for workflows which involve instantiating a new version of a module alongside an old one temporarily during a migration.

However, it does seem like the similar overloading/ambiguity is getting in the way here: you are trying (reasonably) to use the module version argument to specify the exact module version you are currently using, as opposed to which versions of the module your caller is compatible with, and so therefore each time you want to upgrade to a newer version you need to update every single module call to select a different exact version.

If there were hypothetically also a lock-file-like mechanism for module calls then the version argument in a module block could be used only to capture the compatibility information -- e.g. "I require at least version 2.0 of this module" -- and then the actual version selection could be made separately and updated systematically in a central location. The minimum functionality would be "upgrade all modules to the latest available version that's in the compatible set", as terraform init -upgrade does for providers today, but this could also include more specific commands for surgically upgrading particular modules or subsets of modules.

We already have #31301 open to represent the idea of tracking module version selections in the lock file, and so if that is the direction we end up taking here then this issue might end up considered a duplicate of that one. However, that doesn't feel like it's a clear decision to me yet, so I'm just going to create the link between the two issues for now and then we can revisit later on whether it makes sense to consolidate these together into a single request.

[apparentlymart]

In my personal blog (so: not official HashiCorp statements) I recently wrote an article about why I don't think Terraform should support fully-dynamic module source addresses, which is not directly related to what this issue is discussing but does overlap with an alternative idea I discussed in there: allowing to override the installation location for a module registry or provider registry source address in a "dependency override file".

The overall concept is that our registry-style source addresses already serve as an abstraction that separates the physical source location from the name used to talk about the dependency, but today that mapping between logical address and physical address is (for modules) controlled exclusively by the origin module registry. I've proposed to allow the user to override that mapping locally.

However, that idea does still assume that module blocks will specify a version constraint specifying which versions of the module the calling module is compatible with. As I mentioned in my earlier comment, I think we'd also need to support modules in the dependency lock file to also remember which specific version was selected. This issue also makes me wonder about offering a further ability for the "override file" to specify not only where to find the source package for particular versions of a module package but also to impose additional tighter version constraints beyond what the module already specifies.

That would then, along with serving other use-cases, allow a solution to the specific use-case in this issue: an author could choose to leave the version constraint in a module block completely unspecified -- effectively declaring that any version is acceptable -- but then use a dependency override file to give a more specific version constraint. The dependency override file would then be the "central place" as far as folks with this requirement are concerned, but folks who don't have this requirement would be free to continue specifying their version constraints inline if they want.