Nested Maps of Objects in Terraform

June 26, 2022

I alluded to having yet to implement a nested map of objects in a map of objects in Terraform in this post. So, this post will outline my attempt to do so since the need arose and doing so would mesh perfectly with the current module structure that was in use.

Recently, I encountered a scenario where a single object in a map(object({})) variable would result in a resource being created conditionally if present but did not want to implement an overly-complicated way of doing so.

More specifically, an Aurora cluster module would need to create a parameter group resource IIF the specific cluster map object had each.value.parameter_group defined.

The nested variable would look something like this:

variable "cluster" {
  type = map(object({
  ...
  parameter_group = optional(map(object({
     a = optional(string)
     b = optional(string)
     d = optional(string)
     d = optional(list(map(string)))
   })))
  ...
}))

And the DB object would look like this:

db = {
  ...
  parameter_group = {
    db = {
      cluster_identifier = "cluster-name"
      family      = "db-family-here"
      description = "Description here"
      parameters = [
        {
          ...
        },
        {
          ...
        },
        {
          ...
        },
      ]
    }
  }
  ...
}

It’s not critical to note for this example, but Terragrunt is being employed here, so the input is passed in via inputs = {} in terragrunt.hcl.

While this input is the best-case scenario since parameter_group is defined, the other DB objects in the inputs may not require the parameter_group object.

How can this single parameter group be created conditionally and also be created specifically for this cluster?

Not that PGs need to be created 1:1 for a cluster, but let’s assume that it is the case here.

In the module code, the following can be done (the unrelated objects have been omitted) with two local variables:

locals {
  cluster = defaults(var.cluster, {
    ...
    # this is an empty map by default 
    # unless overridden in `inputs`
    parameter_group = {}
    ...
  })

  parameter_groups = {
    for k, v in local.cluster :
    k => v.parameter_group if length(v.parameter_group) > 0
  }
}

In these local variables, the experimental optional feature is used to define object default values (which is extremely helpful for maps with numerous objects) and then local.parameter_groups is used to determine which Aurora clusters actually need a parameter group created.

The optional feature is also good for omitting objects that are not explicitly needed in each map object which can really cut down on SLOC.

Ensure that these values are either handled as above or with ternaries in the resources themselves.

Then, the aws_rds_cluster_parameter_group resource employs the usual for_each with local.parameter_groups to create the correct PGs:

resource "aws_rds_cluster_parameter_group" "parameter-group" {
  for_each    = toset(keys(local.parameter_groups))
  ...
  dynamic "parameter" {
    for_each = local.cluster[each.value].parameter_group[each.value].parameters

    content {
      name         = parameter.value.name
      value        = parameter.value.value
      apply_method = parameter.value.apply_method
    }
  }
  ...
}

Usually, each.key would be used to access the map key when for_each is employed, but the structure of toset(keys(local.parameter_groups)) allows each.value to be used instead; nothing major, just something to note.

The most important thing to note here is that the key used in the parameter group must match the key used in the parent map key (e.g., var.cluster[each.key].parameter_group[each.key].argument), which, again, looks like:

db = {
  ...
  parameter_group = {
    db = {}
  }
}

While Terraform is fairly powerful nowadays compared to its releases (especially prior to 0.13), handling complicated requirements like this is still a little tougher than it should be. Ternaries go a long way with alleviating some of these headaches, but proper support for conditionals and more complex logic would receive no complaints from me.

As an aside, Terraform 1.3.0 finally adding official support for default values to optional objects/variables is extremely welcome. More can be found out about this on the Terraform releases page here.

At any rate, nested maps of objects in maps of objects are a thing, do work with a little bit of engineering, and can still work nicely with for_each.