Simplifying environments

The environments folder remains but now in a much more simplified form. Previously, there were sub-folders for each environment and each region, with a single main.tf file representing the top level of the system. This resulted in a large amount of bloat, a significant amount of duplicated code, plus the one file per folder code smell.

Example of previous environment folder structure

The reason we started with one file per environment and region was to allow us to set specific variables and configure the Terraform backend to store state.

Example of the previous environment and region specific main.tf

In the new design, the death by folders environment structure is gone and each environment is represented by a single JSON file. This means that specific variables can still be set and the entry point is now a single main.tf in the project route which makes things a lot clearer.

Example of the new environment agnostic main.tf in project root

Environment specific variables represented using json

As our infrastructure is the same in each region there is also no need for region specific variables which further reduces duplication. The region was already being passed into the go.ps1 script, to specify the Azure region when running the Terraform commands in the build pipeline, so this parameter can now also be passed into the Terraform plan as a variable to create unique resource names based on our naming conventions.

Example of new environment folder structure

This change was made possible by shifting the responsibility of initialising the backend from the environment main.tf file to the go.ps1 script. The Terraform init command provides functionality to set up the backend using parameters which we extract from the environment JSON.

Executing terraform init using parameters

The Terraform plan is able to accept JSON as a variable file, so that same environment JSON that’s used to extract the backend parameters can be used in the plan. Having a single entry point guarantees the same blueprints are applied to every environment. These changes also allowed us to remove over 250 lines of code and make it easier to maintain, thanks to reduced duplication.

Executing Terraform plan using a variable file

Terraform does provide a concept called workspaces which can be used to control variables for different configurations. I personally feel that workspaces can quickly become quite cluttered, especially if you need to maintain a number of environments. I also don’t think they work so well in automation so we decided to go with the approach above.