Grolifant : A Library to Support Gradle Plugin Development

Compatibility

As of version 0.3, the minimum version of Gradle that is supported by Grolifant is 3.8.

Bootstrapping

The library is available on JCenter. Add the following to your Gradle build script to use it.

repositories {
  jcenter()
}

Adding Grolifant as a compile dependency

dependencies {
  compile 'org.ysb33r.gradle:grolifant:0.3.1'
}

Distribution Installer

There are quite a number of occasions where it would be useful to download various versions SDK or distributions from a variety of sources and then install them locally without having to affect the environment of a user. The Gradle Wrapper is already a good example of this. Obviously it would be good if one could also utilise otehr solutions that manage distributions and SDKs on a per-user basis such as the excellent SDKMAN!.

The AbstractDistributionInstaller abstract class provides the base for plugin developers to add such functionality to their plugins without too much trouble.

Getting started

Extend AbstractDistributionInstaller

TestInstaller.groovy

class TestInstaller extends AbstractDistributionInstaller {
        static final String DISTPATH = 'foo/bar'
        static final String DISTVER  = '0.1'

        TestInstaller(Project project) {
            super('Test Distribution',DISTVER,DISTPATH,project) (1)
        }

        @Override
        URI uriFromVersion(String version) { (2)
            TESTDIST_DIR.toURI().resolve("testdist-${DISTVER}.zip") (3)
        }
}

1	The installer needs to be provided with a human-readable name, the version of the distribution, a relative path below the installation for installing this type of distribution and a reference to an exiting Gradle `Project` instance.
2	The `uriFromVersion` method is used to returned an appropriate URI where to download the specific version of distribution from. Supported protocols are all those supported by Gradle Wrapper and includes `file`, `http(s)` and `ftp`.
3	Use code appropriate to your specific distribution to calculate the URI.

The download is invoked by calling the getDistributionRoot method.

The above example uses Groovy to implement an installer class, but you can use Java, Kotlin or any other JVM-language that works for writing Gradle plugins.

How it works

When getDistributionRoot is called, it effectively uses the following logic

File location = locateDistributionInCustomLocation(distributionVersion) (1)

if(location == null && this.sdkManCandidateName)  { (2)
    location = getDistFromSdkMan()
}

location ?: getDistFromCache() (3)

1	If a custom location location is specified, look there first for the specific version
2	If SDKMAN! has been enabled, look if it has an available distribution.
3	Try to get it from cache. If not in cache try to download it.

Marking files executable

Files in some distributed archives are platform-agnostic and it is necessary to mark specific files as executable after unpacking. The addExecPattern method can be used for this purpose.

TestInstaller installer = new TestInstaller(project)
installer.addExecPattern '**/*.sh' (1)

1	Assuming the `TestInstaller` from Getting Started, this example will mark all shell files in the distribution as executable once the archive has been unpacked.

Patterns are ANT-style patterns as is common in a number of Gradle APIs.

Search in custom locations

The locateDistributionInCustomLocation method can be used for setting up a search in specific locations.

For example a person implementing a Ceylon language plugin might want to look in the ~/.ceylon folder for an existing installation of a specific version.

This optional implementation is completely left up to the plugin author as it will be very specific to a distribution. The method should return null if nothing was found.

Changing the download and unpack root location

By default downloaded distributons will be placed in a subfolder below the Gradle user home directory as specified during construction time. It is possible, especially for testing purposes, to use a root folder other than Gradle user home by setting the downloadRoot

Utilising SDKMAN!

SDKMAN! is a very useful local SDK installation and management tool and when specific SDKs or distributions are already supported it makes sense to re-use them in order to save on download time.

All that is required is to provide the SDKMAN! candidate name using the setSdkManCandidateName method.

Utilising SDKMAN!

installer.sdkManCandidateName = 'ceylon' (1)

1	Sets the candidate name for a distribution as it will be known to SDKMAN!. In this example the Ceylon language distribution is used.

Checksum

By default the installer will not check any values, but calling setChecksum will force the installer to perform a check after downloading and before unpacking. It is possible to invoke a behavioural change by overriding verification.

TestInstaller installer = new TestInstaller(project)
installer.checksum = 'b1741e3d2a3f7047d041c79d018cf55286d1168fd6f0533e7fae897478abcdef'  (1)

1	Provide SHA-256 checksum string

Only SHA-256 checksums are supported. if you need something else you will need to override verification and provide your own checksum test.

Advanced: Override unpacking

By default, AbstractDistributionInstaller already knows how to unpack ZIPs and TARs of a varierty of compressions. If something else is required, then the unpack method can be overridden.

Advanced: Override verification

Verification of a downloaded distribution occurs in two parts:

If a checksum is supplied, the downloaded archive is validated against the checksum. The standard implementation will only check SHA-256 checksums.
The unpacked distribution is then checked for sanity. In the default implementation this is simply to check that only one directory was unpacked below the distribution directory. The latter is effectively just replicating the Gradle Wrapper behaviour.

Once again it is possible to customise this behaviour if your distribution have different needs. In this case there are two protected methods than can be overridden:

verifyDownloadChecksum - Override this method to take care of handling checksums. The method, when called, will be passed the URI where the distribution was downlaoded from, the location of the archive on the filesystem and the expected checksum. It is possible to pass null for the latter which means that no checksum is available.
getAndVerifyDistributionRoot - This validates the distribution on disk. When called, it is passed the the location where the distribution was unpacked into. The method should return the effective home directory of the distribution.

In the case of getAndVerifyDistributionRoot it can be very confusing sometimes as to what the distDir is and what should be returned. The easiest is to explain this by looking at how Gradle wrappers are stored. For instance for Gradle 3.0 the distDir might be something like ~/.gradle/wrapper/dists/gradle-3.0-bin/2z3tfybitalx2py5dr8rf2mti/ whereas the return directory would be ~/.gradle/wrapper/dists/gradle-3.0-bin/2z3tfybitalx2py5dr8rf2mti/gradle-3.0.

Helper and other protected API methods

getProject provides access to the associated Gradle Project object.
listDirs provides a listing of directories directly below an unpacked distribution. It can also be used for any directory if the intent is to see which child directories are available.
getLogger provides access to a simple stdout logger.

Tool Executions Tasks and Execution Specifications

Gradle script authors are quite aware of Exec and /org/gradle/api/tasks/JavaExec.html[JavaExec] tasks as well as the projects extensions exec and javaexec. Implementing tasks or extensions to support specific tools can involve a lot of work. This is where this set of abstract classes come in to simplify the work to a minimum and allowing plugin authors to think about what kind of tool functionality to wrap rather than implementing heaps of boilerplate code.

Wrapping an external tool within a gradle plugin usually have three components:

Execution specification
Project extension
Task type

How to implement these components are described in the following sections.

Execution specifications

Execution specifications are used for configuring the necessary details for runnign an external process. The latter will then be used by a task type of a project extension.

There is currently three abstract classes in the hierachy and the all implement the BaseExecSpec interface.

These execution specifications allow to easily present configuration options to your plugin users such as the following:

Common declarative settings

ignoreExitValue true  (1)
standardOutput System.out  (2)
standardInput System.in    (3)
errorOutput System.err     (4)
workingDir '.'     (5)

1	Whether the exit value can be ignored.
2	Where standard output should be sent to. (It is up to a plugin author to decide on behaviour if this value is `null`).
3	Where standard input is read from. (It is up to a plugin author to decide on behaviour if this value is `null`).
4	Where error output should be sent to. (It is up to a plugin author to decide on behaviour if this value is `null`).
5	The working directory during execution. This is a lazy-evaluated value and can be anything that `project.file()` will be able to process.

Setting process environment

environment = [ foo : 'bar']               (1)
environment foo2 : 'bar2', foo3 : {'bar3'} (2)
environment 'foo4', 'bar4'   (3)

1	Explictly set the environment in an assignment style, removing any previous environment settings.
2	Add additional environment settings in the familiar, and gradlesque, map-style. Values of environmental variables have the ability to be lazily-evaluated by the consuming task or projext extension. (As a plugin author you should consider using `MapUtils.stringizeValues` for your conversions. The tasks described further down do the same).
3	Add one environment setting as a pair of environment variable and its value.

If you are familiar with the options on the Exec task, then the above will come as no surprise. It will also present your plugin user with a familiar set of configuration options.

The executable can also be set in the normal way, but if you set the executable in an implementation-specific way in your implementation you might want to prevent the user from setting executable. These specifications also allows you to provide arguments that is specific to the executable and not any associated command. For instance if you were to do git -C /foo commit myfile.txt, then -C /foo would be executable arguments.

Setting executables and executable arguments

executable {'/path/to/exe'}         (1)
exeArgs = [ 'first', 'second' ]     (2)
exeArgs 'third', {'fourth'}  (3)

1	Set the executable. This is also a lazy-evaluated value and anything that StringUtils.stringize can deal with can be used. In addition ResolvedExecutable instantiations can also be used.
2	Explictly set the execution arguments in an assignment style, removing any previous execution arguments.
3	Add additional execution arguments. All of these values are lazily evaluated.

The above distinction of using execution arguments might seem to be an unnecessary extra at a first read, but in terms of a DSL they allow the use to customise certain behaviour of the excutable without losing focus on the real work the executable is supposed to do. This is similar to running an additional JVM via JavaExec. In this case the `jvmArgs customises the JVM, and not the arguments passed to the class to be executed.

In addition to those the AbstractToolCommandExecSpec will allow you to specify a command that is associated with the executable. For instance in git commit, the command will be commit.

Setting a command and command arguments (AbstractToolCommandExecSpec)

command 'install'         (1)
cmdArgs = [ 'aye', 'bee'] (2)
cmdArgs 'cee', {'dee'}    (3)

1	Set the command. This can be lazy-evaluated.
2	Explictly set the command arguments in an assignment style, removing any previous command arguments.
3	Add additional command arguments. All of these values are lazily evaluated.

In a similar fashion AbstractScriptExecSpec offers the ability to specify a script name and script arguments.

Setting a script and script arguments (AbstractScriptExecSpec)

script 'install.py'       (1)
scriptArgs = ['aye']      (2)
scriptArgs 'cee',{'dee'}  (3)

1	Set the script. This can be lazy-evaluated.
2	Explictly set the script arguments in an assignment style, removing any previous script arguments.
3	Add additional script arguments. All of these values are lazily evaluated.

In order to implement your own execution specification you need to derive from the appropriate specification.

Wrapping Git as a tool with commands

class GitExecSpec extends AbstractCommandExecSpec {
    GitExecSpec(Project project,Object exe) {
        super(project)
        setExecutable (exe ?: 'git')
    }
}

Wrapping Perl as a tool which executes scripts

class PerlScriptExecSpec extends AbstractScriptExecSpec {
    PerlScriptExecSpec(Project project,Object exe) {
        super(project)
        setExecutable(exe ?: 'perl')
    }
}

Creating a project extension

ExtensionUtils.addProjectExtension is the key method to use.

Assume for the moment that you have created a execution specification class for wrapping Git and that it looks like the following:

class GitExecSpec extends AbstractCommandExecSpec {
    GitExecSpec(Project project) {
        super(project)
        setExecutable('git')
    }
}

To add this to

void apply(Project project) {
  ExtensionUtils.addExecProjectExtension('gitexec', project, { Project project ->
      new GitExecSpec(project)
  } as ExecSpecInstantiator<GitExecSpec>) (1)
}

1	An instantiator is required to create instances of execution specifications on demand. One simple way is to create a closure an coerce it to a ../api/org/ysb33r/gradle/olifant/ExecSpecInstantiator.html[ExecSpecInstantiator]

Now it will be possible to do

build.gradle

task gitdiff {
  doLast {
    project.gitexec {
      command 'diff'
    }
  }
}

Creating a task

There is currently three abstract task classes in the hierachy.

The minimum you will need to do is extend the appropriate class and provide a suitable constructor that can be called by Gradle.

Operating System

Many plugin developers are familiar with the OperatingSystem internal API in Gradle. Unfortunately this remains an internal API and is subject to change.

Grolifant offers a similar public API with a small number of API differences:

No getFamilyName and getNativePrefix methods. (A scan of the Gradle 3.2.1 codebase seem to yield to usage either).
No public static fields called WINDOWS, MACOSX etc. These are now a static field called INSTANCE on each of the specific operating system implementations.
getSharedLibrarySuffix and getSharedLibraryName have been added.
Support for NetBSD.

Example

OperatingSystem os = OperatingSystem.current() (1)
    File findExe = os.findInPath('bash')

1	Use `current()` to the operating system the code is being executed upon.

Operating system detection

The logic in 0.3.1 to determine an operating system is

static OperatingSystem current() {
    if (OS_NAME.contains("windows")) {
        return Windows.INSTANCE
    } else if (OS_NAME.contains("mac os x") || OS_NAME.contains("darwin") || OS_NAME.contains("osx")) {
        return MacOsX.INSTANCE
    } else if (OS_NAME.contains("linux")) {
        return Linux.INSTANCE
    } else if (OS_NAME.contains("freebsd")) {
        return FreeBSD.INSTANCE
    } else if (OS_NAME.contains("sunos") || OS_NAME.contains("solaris")) {
        return Solaris.INSTANCE
    } else if (OS_NAME.contains("netbsd")) {
        return NetBSD.INSTANCE
    } else {
        // Not strictly true, but a good guess
        return GenericUnix.INSTANCE
    }
}

Contributing fixes

Found a bug or need a method? Please raise an issue and preferably provide a pull request with features implemented for all supported operating systems.

String Utilities

Converting objects to strings

Use the stringize method to convert nearly anything to a string or a collection of strings. Closures are evaluated and the results are then converted to strings.

StringUtils.stringize( 'foo' )           == 'foo'
StringUtils.stringize( new File('foo') ) == 'foo'
StringUtils.stringize( {'foo'} )         == 'foo'

StringUtils.stringize(['foo1',new File('foo2'),{'foo3'}]) == ['foo1','foo2','foo3']

URI Utilities

Converting objects to URIs

Use the urize method to convert nearly anything to a URI. If objects that have toURI() methods those methods will be called, otherwise objects that are convertible to strings, will effectively call toString().toURI(). Closures will be evaluated and the results are then converted to URIs.

UriUtils.urize( 'ftp://foo/bar' )        == new URI('ftp://foo/bar')
UriUtils.urize( new File('/foo.bar') )   == new File('/foo.bar').toURI()
UriUtils.urize( {'ftp://foo/bar'} )      == new URI('ftp://foo/bar')
UriUtils.urize( {new File('/foo.bar')} ) == new File('/foo.bar').toURI()

What’s in a name

Grolifant is a concatenation of Gr for Gradle and olifant, which is the Afrikaans word for elephant. The latter is of course the main part of the current Gradle logo.