Starting out with Terraform pre-commit

TL:DR; Follow these instructions in Linux and WSL2. Basic config file here. Windows is not directly suitable for running pre-commit, IMHO. Running containerized can be a solution.

Please consider sponsoring Anton Babenko if you find pre-commit-terraform useful (container image and repo)! I am not in any way affiliated, and will not gain anything from this. His hard work on community projects deserve some reward, IMO.

I started looking into pre-commit hooks for a project a while ago, but haven’t used it much since. A new project coming up is utilizing it, and I wanted to get into it again. What better way to get a basic understanding of it than trying to explain it here?

What is pre-commit?

Pre-commit is a set of tasks that can be run before you commit a change to git.

“[Pre-Commit] automatically points out issues in code such as missing semicolons, trailing whitespace, and debug statements. By pointing these issues out before code review, this allows a code reviewer to focus on the architecture of a change while not wasting time with trivial style nitpicks.“

It is also a great way to automatically document your Terraform code in markdown! It is important to note that in this post I will focus on Anton Babenko’s pre-commit-terraform, and not only the generic service. They are somewhat integrated with each other.

Benefits of usage?

There are many benefits of using pre-commit hooks, but these are some of them listed:

Syntax validation before code review
Automatic docs generation for Terraform
Terraform linter
Security and compliance scanning with tfsec and checkov, among others
Whitespace trim
Line endings change
Cost estimation before infrastructure deployment with Infracost

Attempted use in Windows

I tried using the python version of pre-commit, but I couldn’t get it to work at first. Installed Python 3.10 from winget and from windows store, but it wouldn’t run at all. Then I installed Python manually from this web page (Installer amd64).

Now it starts, but it’s clear to me that this requires lots of fiddling to work correctly. The requirements must be installed manually (tflint, tfdocs, tfsec, checkov ++), and I just want this to work. Also I don’t want another software to update regularly.

Use in Ubuntu 20.04

Have Ubuntu 20.04 (either WSL on Windows or Linux client)
Install Python 3.10 (or latest version)
Use this guide for the rest.
Install the prereqs you need as pointed out in the Ubuntu 20.04 instructions.
Navigate to your terraform folder, where the pre-commit config file is placed, and do pre-commit run -a.

As far as I can see, the Linux usage of pre-commit is much more streamlined than Windows. You install with the provided cmds.
This is how I installed in my WSL Ubuntu, and it works like a charm! 💪🐧

Config file

To run pre-commit you need a config file, with filename .pre-commit-config.yaml. This is a hidden file you create in the repository where you want pre-commit to run. There are many more complex ways of running, but I won’t go through them here.

You can copy the simple config file from my repo, the one from pre-commit-terraform repo, or the sample:

repos:
- repo: https://github.com/antonbabenko/pre-commit-terraform
  rev: <VERSION> # Get the latest from: https://github.com/antonbabenko/pre-commit-terraform/releases
  hooks:
    - id: terraform_fmt
    - id: terraform_docs

The repository mapping tells pre-commit where to get the code for the hook from. Then you choose a version, and you enable some hooks. The example would run hook for terraform_fmt to format your code, and terraform_docs to create/update a markdown doc file. You can grow the file from there, if you need.

Install git hooks

Run pre-commit install in your git repository to ensure automatically running before each commit. This is a fairly straightforward process. If you don’t install this way, you must run pre-commit manually before each commit.

Use in workflows

Pre-commit will run automatically if you install the hooks with pre-commit install. This will update documentation, and fix your syntax. It will also validate terraform to a degree with tflint, tfsec, and checkov, if you have enabled this in the config file.

You might need to stage twice, if files are updated by pre-commit. If not, you can get an error:

If this happens, just stage files again (git add -A / git add .), and re-commit.

There are several options for running this in CI/CD, and I will mention a subset here.

Run the pre-commit-terraform docker image
- This is the approach I have chosen at the moment. It is simple enough. It requires little configuration on my part, other than the pre-commit-config file. It does not require any maintenance on my part. This is my very basic workflow for running pre-commit on all pushes to main branch. Please consider sponsoring Anton Babenko if using this. He does great work, among other contributors in this repository!
Pre-commit.ci-service
- Pre-commit.ci is maybe the easiest, but also it’s a separate service.
- Free for personal use in public repositories. Results from runs pops up in a separate web page other than GitHub. I felt this gave less control, but might try it again some time.
Manual install
- Install pre-commit and all requirements like in Linux for each run. This is a full control approach, but will take time for each run.
- Basically requires you to install all tools each time the job is run. Lots of wasted time there. You can find a simple example here.
Build custom docker image
- You can create a custom docker image with the required tools, and run this. This is a fast approach for each run, but you will need to maintain your docker image. Shamelessly stole the dockerfile from this medium post (behind paywall).

Alternatives in Windows

Docker Desktop is a possibility if your company has purchased a license to use it. Recent changes to Docker Desktop licensing requirements has made it unavailable to me on my work laptop.

WSL2 is a great way of running Linux in your Windows installation. I use this all the time, and now I am using it for running pre-commit on Terraform code. This has been the easiest method for me, but it requires configuring WSL2 before use.

Try to install Python, pre-commit, and all required prerequisites. Keeping them up to date, with Python version and all. Chocolatey can maybe help with this, but it is not a tool I am using. Winget did not work as expected for this.

Summary

I will say in conclusion that using pre-commit is a great way to improve your IaC workflow. Where possible I will try to use this going forward, and with WSL2 this is made easier. Also the VSCode Remote WSL extension simplifies things.

Stay tuned for more IaC snacks going forward! And please leave a comment if you find any of this useful, or see any errors/misconfigurations.