Introduction

Like many software projects, frameworks, plugins they start off because the author could not find a solution that really fitted their needs and submitting pull requests to existing projects would not have changed the direction of such projects to the needs of the author. This is one of those kind of projects and it is offered against a number of alternative solutions that are available as Gradle plugins.

The aim with the group of plugins is making the integration of Terraform into the build automation pipeline as smooth as possible. This combination of Gradle with Terraform makes for an extremely powerful orchestration platform in the DevOps.

The plugin brings with it a number of subgoals:

  • Simplicity to use defaults - convention over configuration

  • Maximum flexibility if you need it.

  • No need to install terraform or relevant tools - let Gradle take care of it for you.

This is an incubating project. Until is 1.0 released one day, interfaces and DSL may change between 0.x releases.
These documentation pages assume that you have at least a basic working knowledge of Terraform.

Terraform + Gradle - A mindset change

Integrating these two products delivers a pwerful combination, but also presents a number of operational ways that challenges people that is used to using terraform standalone or in scripts.

Using Terraform with Gradle means that the former no longer needs to be installed. Just specify the version and Gradle will boostrap the correct version. If the version is already in cache (~/gradle/native-binaries/), Gradle will just reuse it.

Caching of plugins

Gradle will cache Terraform plugins using the Terraform caching mechanism, but share it between project by using the Gradle user home caching directories. It is possible to override this a have the caching use a per-project basis or purely rely on global Terrafom settings.

Bootstrapping

These plugins are available from the plugin portal. Add the appropriate plugin identifiers to your build.gradle file depending on the type of functionality you require.

build.gradle
plugins {
  id 'org.ysb33r.terraform.base'    version '0.4.0'  (1)
  id 'org.ysb33r.terraform'         version '0.4.0'  (2)
  id 'org.ysb33r.terraform.rc'      version '0.4.0'  (3)
  id 'org.ysb33r.terraform.wrapper' version '0.4.0'  (4)
}
1 Base plugin provides terraform extension.
2 Conventions for terraform including source sets and tasks created by convention.
3 This plugin is normally not applied directly, but deals specifically with Terraform configuration
4 This plugin is usually only applied at the root project and dealks with the creation of terraformw wrappers.
You need at least Gradle 4.7 to use these plugins.

Base plugin

The base plugin provides:

  • A project extension named terraform.

  • Various Terraform task types which tend to map the terraform command closely. For instance for the init command, the appropriate task type is called TerraformInit.

  • Ability to download and use terraform executables.

When the base plugin is applied to the root project it will automatically apply the terraformrc plugin.

Terraform plugin

The Terraform conventions plugin provides:

  • terraformSourceSets as a project extension.

  • The default Terraform source set called main with default directory src/tf/main.

  • A number of tasks including tfInit, tfPlan and tfApply which acts upon the default source set.

Applying the Terraform plugin will apply the base plugin.

Terraform RC plugin

The terraformrc plugin provies an extension called terraformrc which deals with the creation of a configuration file specificaly for use by Terraform within the project.

Terraform Wrapper plugin

Provides the terraformWrapper and cacheTerraformBinaries tasks.

Quick start

The minimalist project is

plugins {
  id 'org.ysb33r.terraform' version '{revnumber}'
}

Now add a init.tf file to src/tf/main and add some Terraform content to it.

$ ./gradlew tfInit (1)
$ ./gradlew tfApply (2)
$ git add src/tf/main/terraform.tfstate (3)
1 Initialise your Terraform project environment.
2 Apply your changes
3 Add your newly created .tfstate file to source control. If your Terraform context uses remote state storage, then this last step is not required.

Platform installation support

These plugins can automatically download, cache and render Terraform for the following platforms:

  • Linux 32 & 64-bit.

  • Mac 64-bit.

  • Windows 32 & 64-bit.

  • FreeBSD 32 & 64-bit.

  • Solaris 64-bit.

Should you need to run Gradle on a platform not listed above, but which Terraform supports and on which Gradle can run, you will need to configure the Terraform executable via the path or search methods. You can also raise an issue to ask for the support or you can submit a PR with the solution.

Source Sets

When the org.ysb33r.terraform plugin is applied, it adds the terraformSourceSets extension which contain the main source set. Associated with this source set is the default folder of src/tf/main and a number of tasks including:

  • tfApply

  • tfDestroy

  • tfImport

  • tfInit

  • tfPlan

  • tfShowState

  • tfValidate

If additional source sets are needed they can be added by convention i.e.

terraformSourceSets {
    development (1)
    staging {
        srcDir = 'staging' (2)
    }
}
1 Creates a Terraform source set named 'development' with default directory src/tf/development
2 Creates a Terraform source set named 'release` and set the directory to staging.

Tasks for additional source sets follow the tf<SourceSetName><TerraformCommand> format. For instance in the above example the initialisation task for the development source set will be called tfDevelopmentInit.

By convention all tasks that map Terraform commands start with tf. Other non-commands tasks might start with terraform or contain Terraform within the task name.

Terraform extension

The terraform project extension configures a method of obtaining a terraform executable. On supported platforms it offers the option of configuring a Terraform version, which Gradle will then download, cache and use. THe project extension also allows for the definition of variables at a global level.

Each Terraform task also has a terraform extension which can be used to override any settings from the project extension on a per-task basis.

Terraform variables

A number of Terraform task types support vairtables and files containing variables. Any variables block support the following functionality

variables {
    var 'foo', 'bar' (1)
    map 'fooMap', foo: 'bar' (2)
    list 'fooList', 'foo', 'bar' (3)
    list 'fooList', [ 'foo', 'bar'] (4)
    file 'filename.tf' (5)
}
1 Adds one variable called foo or value bar. The provided value can be anything that can be lazy-evaluated to a string.
2 Adds a map called fooMap. The map keys have to be strings, but the map values can be anything that will lazy-evaluate to a string.
3 Adds a list called fooList with values foo and bar. THe list entries can be anything that will lazy-evaluate to a string.
4 Alternative list format
5 Adds a file containing a list of terraform variables. The name can be anything that will lazy evaluate to a string and may be a relative path. The file will be resolved relative to the source set directory.

There a are a number of ways to define these depending on the context.

On project extension

When variables are defined on the terraform project extension they are visible to all Terraform tasks that want to utilise them.

On source set

When variables are defined on a specific Terraform source set they are visible to all Terraform tasks that are associated with the specific source set.

On task extension

When variables are defined on the terraform task extension, they are only visible to the specific task. A task extension can also be set up to ignore any variables from the project extension of the associated source set.

terraform {
    variables {
      var 'foo', 'bar' (1)
    }
}

terraformSourceSets {
    main {
        variables {
            var 'foo', 'bar' (2)
        }
    }
}

terraformPlan {
    terraform {
        variables {
            var 'foo', 'bar' (3)

            global.ignore = true (4)
            sourceSet.ignore == true (5)
        }
    }
}
1 Adds a global terraform variable.
2 Adds a source set-specific terraform variable.
3 Adds a task-specific terraform variable.
4 Ignore any global terraform variables.
5 Ignore any source set-specific terraform variables.

Environmental Variables

Gradle execution environment is not passed down to Terraform by default with the exception of specific platform-dependent variables as defubded by the following logic:

if (OS.windows) {
    [
        TEMP        : System.getenv('TEMP'),
        TMP         : System.getenv('TMP'),
        HOMEDRIVE   : System.getenv('HOMEDRIVE'),
        HOMEPATH    : System.getenv('HOMEPATH'),
        USERPROFILE : System.getenv('USERPROFILE'),
        (OS.pathVar): System.getenv(OS.pathVar)
    ] as Map<String, Object>
} else {
    [
        (OS.pathVar): System.getenv(OS.pathVar)
    ] as Map<String, Object>
}

Environmental variables can be defined at two levels:

  • Project level via terraform project extension

  • On the task itself.

terraform {
    environment = [:] (1)
    setEnvironment([:]) (2)
    environment AWS_SHARED_CREDENTIALS_FILE : '/path/shared.creds' (3)
}

terraformInit {
    environment = [ AWS_SECRET_ACCESS_KEY : '12345' ] (4)
    setEnvironment([]) (5)
    environment AWS_SHARED_CREDENTIALS_FILE : '/path/shared.creds' (6)
}
1 Clear all project environment settings and replace with empty set.
2 Alternative setter.
3 Add a project-level environmental variable.
4 Clear all task-level environmental variables (in this example the tfInit task) and replace with a new set. Clearing the environmental variable set will not remove the efault environmental variables. If you need to have them changed, set them explicitly
5 Alternative task-level setter.
6 Add environmental variable to the task only.
It is possible to call the environment method on the task’s terraform extension, but those methods will simply forward to the methods on the task itself.

Command-line options for tasks

A number of task types support command-line options.

Task Option Type Default Purpose

tfApply

--targets

list

 

Select group of resources to apply.

tfDestroy

--targets

list

 

Select group of resources to apply.

--approve

boolean

false

Auto-approve destruction of all mentioned resources.

tfImport

--type

string

 

Type of resource to import e.g. aws_ecs_cluster

--name

string

 

Name of resource to import as specified in .tf file

--id

string

 

Actual identifier to import. Check the Terraform documentation for the specific resource in order to know how to identify this.

tfInit

--upgrade

boolean

false

Force upgrade of modules

--configure-backends

boolean

true

Configure backends

tfPlan

--targets

list

 

Select group of resources to apply.

--json

boolean

false

Write textual plan in JSON format.

tfShowState

--json

boolean

false

Write validation output in JSON format.

tfValidate

--json

boolean

false

Write validation output in JSON format.

Just in case this documentation is out-of-date, always run ./gradlew help --task <taskName> to get a description of supported command-line options.