In CWISE we use Terraform a lot. The most common use cases for Terraform for us is cloud resource provisioning, Kubernetes configuration management, and SaaS services (like Github/Gitlab) management.
We prefer Terraform over many other competitors due multiple reasons:
- Tried and tested tool, has been around for a long time and Hashicorp is doing great work of developing it. Can be defined as mature and even boring technology;
- A large number of community resources like providers, modules, and documentation;
- Good developer experience due to support in IDE's and support tools;
- Has got a configuration state (database);
How We Do Terraform Implementantions
Next, we'll delve into the mechanics of how we at CWISE implement Terraform. Will provide a short overview of our code structuring practices and state configuration strategies, all of which are aimed at achieving efficient and reliable infrastructure management. Also challenges what we face doing in this way.
The preferred approach for code structure is layered, an approach based on logical separations. Each logical layer gets its own code folder and own backend configuration For example, the IAM code is located separately from the Network code.
IMPORTANT note: we try to leverage as many modules (especially public ones) as possible, the environment config is composed of modules, not bare resources. IMPORTANT note: most of our executed projects are medium-size to large-size, medium-high complexity projects in regulated industries
Benefits of code and remote state structuring approach
The blast radius is minimized if something goes wrong; For example, if one machine is destroyed, then by accident other items will not be affected;
We find it easier to implement dependency handling, in this way due fact, that you need to finish the previous stage before the next can be run. Like you need to provision networks before, GKE clusters. Or you can provision helm charts only when Kubernetes connection strings are known.
Ability easier implement separation of duties and organize code ownership
Challenges of code and remote state structuring approach
Data sharing between states can be labor intensive
More code and states to handle. As there is more boilerplate code and the state count grows significantly;
Orchestration to execute Terraform code in the correct order can be painful, especially in CI/CD systems;
Challenge 1. is pretty easy to handle, share data thru data sources (preferred) or via remote states (please avoid it as much as you can);
We will focus on cons 2 and 3.
Past Methods Tried to Overcome Challenges to Scale Terraform
Bare Terraform Hacks
Different Terraform hacks, like using dummy data objects for common deno
Terragrunt
Most popular Terraform wrapper. Works on top of Terraform. Many quirks and features. In theory, all should be good but did not stick with us.
The challenges we faced with Terragrunt:
Not always dependencies were executed in the right order;
Lost clear access to Terraform code if implemented in an advised way (only terragrunt.hcl). Even simple actions could become pretty complex;
Executions become very slow;
Most IDE integrations don't work anymore if implemented in an advised way (only terragrunt.hcl);
Different Code Generators
Mostly focused on challenge 2. Projects like ahzhezhe/terraform-generator etc. Why did not this approach stick with us:
Generations get messy fast if more complex variables or locals constructions are used.
Did not find this approach natural workflow for us. Hard to measure, but still
note: did not trialed CDKTF.
Current Approach to Scale Terraform: Terramate
After many trials and errors, we have found, that Terramate solves fully or partially our challenges. It helps us: - To speed up and scale development;
- Reducing overall maintenance for our Clients' implementation;
What is Terramate? Cutting marketing down texts, it's TF code generator, TF code orchestrator, drift detection, and other features. It's heavily built around git. It's a relatively new project, first release in 2021.
Why We Liked Terramate?
It's easy to plug and play with the existing Terraform code base. Skipping the code generation part that is more intrusive, just using the orchestration part is almost zero effort. The steps are super simple. For example, there is an example project not compatible with Terramate. To get started. all that you need is to do is: I. Add "terramate.tm.hcl" file in top of directories II Go thru all directories and issue command: "terramate create {{ directory_name }}" After you can start using orchestration features. https://terramate.io/docs/cli/orchestration/
It's code generator, it's not a wrapper. When code is generated you get plain Terraform code, which can be used without Terraform. It's not hidden somewhere in cache or anywhere else. Code generation is basic but it works and by coupling with globals you can generate a lot.
Orchestration solves two things. It's easy to organize dependency order and it's possible to execute on whole stack (like environment) terraform apply/terraform plan/terraform init etc. And you can execute only changed part in repo. What we saw, that CI/CD environments speed up was significant in some cases even multiple times.
What We Don't Like with Terramate?
Code generation does general stuff, but for example, if you need to generate in one file conditions, we did not find how to do it.
It's a pretty new project, without clear business behind it. I know, that they are testing out the Terramate cloud. But still, HC can come in and just copy pains.
Conclusion
Terramate helps to scale Terraform. It addresses two problems, that have been around since the inception of Terraform:
Generating code, especially parts where templating is not available out of the box for Terrraform itself.
Complex code execution orchestration scenarios in the desired order and with scale
We use it because it's an elegant way how to solve problems of boilerplate and orchestration. When Terramate is used, it's compatible with Terraform.
Compared to other tools, like Terragrunt/Terraspace: think Ansible vs Chef/Puppet analogy. Provides the same value, but a much simpler implementation
Can we recommend this tool: yes. I Hope Terramate team will keep developing in the same pace.