GradleTest is available in the Gradle Plugins repository. To use it, add the appropriate plugin to the plugins block.

When your setup is more complex and the plugins block does not work OR if you are using Gradle 2.0 use the following instead

Dynamic dependencies: Hard-coding the plugin version in to the build.gradle files of the gradleTest test fixtures is a maintenance pain. However the plugin is injected automatically into the build via GradleTestKit.

In build.gradle define the versions of Gradle that needs to be tested:

Create a folder called src/gradleTest. Beneath that add a folder - call it 'myExample` for now. Within that create a normal Gradle project i.e. drop a build.gradle file and fill it out with the kind of syntax that a person will use to run your plugin. Finally add a custom task called runGradleTest and make it depend on some task that your plugin provides.

For example here is one example from GradleTest’s test suite of how itself is being tested.

It is very convenient to sometimes to runs the test with a subset of the versions specified in the configuration. This is achievable by passing a system property to Gradle

gradleTest is linked into the check lifecycle task.

HTML reports will appear as folder named after the GradleTest instance below the ${reporting.baseDir} folder. Practically in most cases this will be build/reports/gradleTest.

Use the additionalGradleTestSet project extension. Specify the basename and all of the appropriate configurations and tasks will be added.

Running lots of compatibility tests can take some time. If you need more visual feedback regarding progress try adding something like this to the gradleTest configuration.

With the growing use of Kotlin-based build scripts, it is also important to test compatibility of a plugin for this context. Turn Kotlin-testing on by setting the following:

Now drop a build.gradle.kts file alongside your build.gradle build script. If the gradle version under test is 4.0 or later, GradleTest will also run a test using the Kotlin build script.

GradleTest is aware of certain command-line parameters and will pass them on to running tests.

If the convention of adding a gradleTest with it’s default settings are not suitable for your environment, then the base plugin can be considered. This will only add the additionalGradleTestSet method. The GradleTest Base Plugin is available in the Gradle Plugins repository. To use it, add the following snippet into your build script.

When your setup is more complex and the plugins block does not work OR if you are using Gradle 2.0 use the following instead

%%.VERSION%% is no longer supported when building with Gradle 2.13+ and builds will fail with Could not find foo:bar:%%VERSION%%` error if encountered. It will still be supported if using Gradle 2.12 or earlier as legacy mode will be invoked. Even under legacy mode it’s use is no longer required. Simply remove the classpath, and if you don’t anything extra in the buildscript block, the whole of the buildscript block.

For instance the following block can be complety removed:

GradleTestKit is the core API that ships with Gradle since Gradle 2.6. which is meant for plugin developers to test compatibility across Gradle versions. Easrly versions could only tests against the same version of Gradle that was running, but it has been enhanced over different Gradle releases. Today it is powereful allowing multiple versions to be tested and a number of different environments to be set up.

It is also a core compoenent of GradleTest when the version of Gradle is 2.13 or later. (When the current build version is older GradleTest falls back to legacy mode, simply because the GradleTestKit API either is non-existent or not mature enough.

GradleTestKit works well for integration testing when writing plugins if a select small group of Gradle versions are used or even if it only targets the current Gradle version. In this way one can quickly determine issues in a plugin which cannot be discovered via normal testing. GradleTest used GradleTestKit directly for its integration tests for exactly this reason.

Testing multiple versions of Gradle with GradleTestKit soon bcomes a burden for developers as the API differs and classpath setups can be tricky. Older versions of Gradle give the most issues. In addition GradleTestKit is essentially a library it focuses on the developer rather than the build script user.

In this regard GradleTest steps in as it allow tests to be written as if a normal buildscript would be written. it also manages differences (or non-existance) in API from Gradle 2.0 onwards in a completely transparent way to the plugin author.

Overall the approach should be to write integration tests with GradleTestKit using a limited or even just one Gradle version. This is then complimented by some GradleTest scripts to test a wider range of Gradle versions.

Stutter is another library by Andy Oberstar to aid with plugin development. It focuses on the cases where you might have to run the same tests against all supported versions, but don’t want to deal with making each test loop over those versions (or @Unroll with a where block in Spock). Stutter generates one task per supported version for you and runs the full set of tests against it.

It definitely removes some of the complexity of setting up the GradleTestKit environment, but beware that it works from Gradle 2.13 onwards. The latter is not a restriction, but essentialy the same reason why GradleTest reverts to legacy mode if the build version of Gradle is older than 2.13: a lack of maturity in the GradleTestKit API in those releases.

It adds a compatTest source set, which will not clash with the gradleTest source set from GradleTest.

Stutter is essentialy developer-centric whereas GradleTest is script author-centric. This distinction is important as it clarifies different levels of validation:

It can be an easy way to manage some more complex plugin integration tests. There is no reason to not use it alongside GradleTest as part of plugin development.

Hard-coding the plugin version in to the build.gradle files of the gradleTest test fixtures is a maintenance pain. In previous versions of the plugin one had to dp to write something like

and the plugin would substitute the %%VERSION%% token with the version of your project. This is no longer necessary with this release even when running in legacy mode. The plugin will inject the path to the plugin automatically eliminating the need to use a block like the above at all.

The gradleTest task is not linked to any other tasks. Run this as explicit task on the command-line or add your own task dependencies in your gradle script. The reason for this is that in legacy mode it can be quite a time-consuming testset to run. The typical case will be that the tests are only run close to release time. If you prefer you might want to set install.dependsOn gradleTest rather than check.dependsOn gradleTest or build.dependsOn gradleTest.

For testing, a folder is created for each gradle version to be tested and the projects are duplicated below the verson folder. Each version’s testsuite is executed within a separate JVM.

It is possible to add additional test tasks beyond gradleTest, by doing

Test files should be placed under src/furtherTest using the same layout as described earlier. Dependencies should be listed under furtherTest configuration.

Global configuration is still read from gradleLocations project extension.

Unresolved directive in product-documentation.adoc - include::parts/gradle-runner.adoc[]