Introduction

Like many software projects, frameworks, plugins they start off because the author could not find a solution that really fitted their needs and submitting pull requests to existing projects would not have changed the direction of such projects to the needs of the author. This is one of those kind of projects and it is offered against a number of alternative solutions that are available as Gradle plugins.

The aim with the group of plugins is making the integration of Node.js and related technologies into the build automation pipeline as smooth as possible. This brought with it a number of subgoals:

  • Simplicity to use defaults - convention over configuration

  • Maximum flexibility if you need it.

  • Removing as many Node.js hurdles from the learning curve as possible.

  • No need to install node or relevant tools - let Gradle take care of it for you.

  • Allow other developers/contributors of your projects to build it with out fiddling with a big list of prerequisites.

  • Isolate builds from potential side effects due to global node installations.

This is an incubating project. Until is 1.0 released one day, interfaces and DSL may change between 0.x releases. The minimum required Gradle version is 4.9.
Quick starts

Alternative solutions

This is not the only solution. You might also want to look at

Bootstrapping

These plugins are available from the plugin portal. Add the appropriate plugin identifiers to your build.gradle file depending on the type of functionality you require.

build.gradle
plugins {
  id 'org.ysb33r.nodejs.base'    version '0.10.0'  (1)
  id 'org.ysb33r.nodejs.npm'     version '0.10.0'  (2)
  id 'org.ysb33r.nodejs.gulp'    version '0.10.0'  (3)
  id 'org.ysb33r.nodejs.cmdline' version '0.10.0'  (4)
  id 'org.ysb33r.nodejs.wrapper' version '0.10.0'  (5)
  id 'org.ysb33r.nodejs.dev'     version '0.10.0'  (6)
}
1 Base plugin
2 Support for using NPM
3 Support for using Gulp
4 Support running commands like node from the command-line with arguments. See running commands from command-line.
5 Create wrappers for node, npm and npx. See creating wrappers.
6 Use Gradle to manage your Node.js project development.

Base plugin

The base plugin provides:

  • nodejs extension.

  • nodeexec execution extension.

  • Ability to download and use Node distributions.

NPM plugin

The NPM plugin provides:

  • npm extension.

  • NpmTask type.

  • npmexec project extension to run NPM commands.

  • npm, npmDevOnly, npmOptional configurations.

  • npmPackage for listing packages in Gradle dependencies blocks.

  • NpmExecSpec.

Applying the NPM plugin will apply the base plugin.

Gulp plugin

The Gulp plugin provides:

  • gulp project and task extensions

  • GulpTask task type.

Applying the Gulp plugin will apply the NPM plugin.

Working with Node

Configure global Node defaults

The Node.js distribution can be supplied in three possible ways:

build.gradle
nodejs {
    executableByVersion('7.10.0') (1)
}

nodejs {
    executableByPath('/path/to/node') (2)
}

nodejs {
    executableBySearchPath('node') (3)
}
1 Supply a version and Gradle will take care of downloading the distribution and caching it. It will then used the cached version for running node. This is the preferred way of running Node.js with Gradle as it offers better control over reproducible builds.
2 Supply a path to the node executable. This works for people that have Node.js installed in standard locations.
3 Tell Gradle to look for node (or node.exe) in the system search path.

If no executable is configured, then Gradle will attempt to download the version specified in NodeJSExtension.NODEJS_DEFAULT. If Gradle runs on a non-supported platform this will fail.

Using nodeexec

It is possible to run node directly from Gradle using the nodeexec project execution extension. It operates in a similar fashion to Gradle’s Exec task, but it has it’s own node nuances, which the user need to setup correctly.

nodejs.exec {
    script 'node_modules/gulp/bin/gulp.js' (1)
    scriptArgs '--help' (2)
    workingDir npm.homeDirectory (3)
    executable nodejs.executable.get() (4)
}
1 Specify the name of the script. It can be absolute of relative to the working directory.
2 Specify any arguments that will be passed to the script.
3 Set the working directory for the execution. If this is something installed via NPM, then using the location of the NPM homedirectory from the npm extension will simplify life.
4 Specify the locaton of the node executable. There are various ways of doig this, but a easy way is to ask the node extension to resolve it.

Consult the NodeJSExecSpec API documentation for full details of all of the settings.

Platform installation support

These plugins can automatically download, cache and render Node for the following platforms:

  • Linux 32 & 64-bit.

  • Mac 64-bit.

  • Windows 32 & 64-bit.

Should you need to run Gradle on a platform not listed above, but which Node supports and on which Gradle can run, you will need to configure the Node executable via the path or search methods. You can also raise an issue to ask for the support or you can submit a PR with the solution.

Working with NPM

Configure global NPM defaults

The location of npm-cli.js can be supplied in four possible ways:

build.gradle
npm {
    executableByPath('/path/to/npm') (1)
}

npm {
    executableBySearchPath('npm') (2)
}
1 Supply a path to the npm executable. This works for people that have npm installed in predefined locations.
2 Tell Gradle to look for npm (or npm.cmd) in the system search path, then infer location of npm-cli.js.
Gradle does not use the npm or npm.cmd scripts. It uses npm-cli.js directly.

By default the default local configuration is always set to an npmrc file located in the root of the root project and the global configuration is set to ~/.gradle/npmrc. This is specifically done so that configuration can be set on a project-wide basis for a Gradle project without interference from a possible globally installed Node.js. Placement of global configuration in the Gradle User Home allows a person to set specific global options for Node projects that are built by Gradle. The location of the global and local npmrc files can also be set.

build.gradle
npm {
  localConfig "${projectDir}/npmrc2" (1)
  globalConfig "${project.rootProject.projectDir}/npmrc2" (2)
  homeDirectory 'src/node' (3)
}
1 Set location of local configuration file.
2 Set location of global configuration file.
3 Set the location of the Node project code. This is the directory into which node_modules will be generated into and per convention the parent directory of the actual Node.js source directories.

NpmTask type

NpmTask is a generic task type for running NPM commands. In its simplest form it takes an npm command and a collection or arguments.

task createPackageJson( type : org.ysb33r.gradle.nodejs.tasks.NpmTask ) {
    command 'init' (1)
    cmdArgs '-f', '-y' (2)
}
1 npm command
2 Arguments that are applicable to the specific command

Customisation

By default an NpmTask will obtain the location of node from the nodejs project extension and any NPM configuration from the npm project extension. However, this plugin allows you to have a lot of flexibility should you need it. Therefore you can override any of the settings in both of the aforementioned extensions in task extensions by the same name. This allows you to for instance run a specific task with a different version of node than the global configured version.

By taking the same previous example, you can do

task createPackageJson( type : org.ysb33r.gradle.nodejs.tasks.NpmTask ) {
    command 'init'
    cmdArgs '-f', '-y'

    nodejs {
      executable version : '7.10.0' (1)
    }

    npm {
      localConfig "${projectDir}/npmrc2" (2)
    }
}
1 Change the specific of node when executing this task.
2 Use a different local configuration file.
This kind of flexibility is not applicable for probably the majority of cases, but there are specific use cases where this is extremely helpful.

Any other parameters that you would expect to see in an Gradle ExecSpec can be used as well.

Working with package.json

There are a number of tasks that work directly with package.json.

Creating a package.json file

NpmPackageJsonInit will create a bare bones package.json file

task packageJsonInit(type: org.ysb33r.gradle.nodejs.tasks.NpmPackageJsonInit) {
   npm {
       homeDirectory = 'src/node' (1)
   }
}
1 Sets location of package.json. If not set it will default to ${npm.homeDirectory}/package.json.

Installing NPM packages

NpmPackageJsonInstall installs packages defined in package.json.

task packageJsonInstall(type: org.ysb33r.gradle.nodejs.tasks.NpmPackageJsonInstall) {
   npm {
       homeDirectory = 'src/node' (1)
   }
}
1 Sets location of package.json. If not set, it will default to ${project.npm.homeDirectory}/package.json.

Synchronising project version to package.json

SyncVersionToPackageJson allows for the project version to be synchronised.

task syncPackageJson(type: org.ysb33r.gradle.nodejs.tasks.SyncVersionToPackageJson) {
   forceSemver = true (1)
   packageJsonVersion = '1.2.3' (2)
   packageJsonFile = 'src/node/package.json' (3)
   npmConfigurations 'npm' (4)
   conflictMode = GRADLE (5)
}
1 Decide whether the project version should be converted to a semver version that is NPM compatible. By default this is set to true and if the version it 1.0 then the version written will be 1.0.0.
2 Set the version to be written. If not set, the project.version will be used by default. The version is lazy-evaluated and can be anything that stringize will convert.
3 Set the location, which is lazy-evaluated and can be anything that fileize will convert This setting is required.
4 If you want to sync dependencies from Gradle into package.json you can add those configurations in here. See Dependency Management for more details on specifying NPM packages in Gradle configurations/
5 If you are providing packages via GRadle you need to say which source to use when there are conflicting package versions. Valid values are GRADLE and PACKAGE_JSON.

Dependency management

The NPM plugin adds a npmPackage extension that can be used inside the dependencies block to add NPM dependencies. For convenience it can be used along with the npm configuration that is also added by the same plugin, however nothing prevents you from adding NPM dependencies to other configurations as well.

dependencies {
    npm npmPackage(name: 'stringz', tag: '0.2.2')
}

Anything added with npmPackage can also update package.json depending on the type property.

Current properties that can be used are:

  • scope - NPM scope

  • name - NPM package name

  • tag - NPM tag

  • type - One of prod, dev, optional. If not supplied, will default to prod.

  • install-args - Additional arguments that NPM has to use during installation.

  • path - Modify the system path when installing this package. By default packages are installed with a very restricted environment. For instance, if the package installation requires access to /usr/bin, then this can be set here.

The transitive logic handling is still pretty crude.

Working with Gulp

Configuring Gulp

Support for working with Gulp is available via the Gulp plugin (org.ysb33r.nodejs.gulp). It adds a gulp extension which is the place to configure global settings for working with Gulp.

build.gradle
gulp {
    executableByVersion('1.2.3') (1)
    gulpFile 'foo/gulpfile.js'   (2)
    requires 'foo', 'bar'        (3)
}
1 Tell Gradle where to find Gulp. The recommended approach is just to specify a version. if nothing is configured gradle will use a default version set with the plugin. See GulpExtension.GULP_DEFAULT.
2 Set the location of gulpfile.js. The default is "${project.projectDir}/gulpfile.js". This causes Gulp to chage working directory to the location of this file. It does however still keep the original working directory from NPM in mind as well.
3 Add additional requires that will be passed as --requires to Gulp.

In most cases the default settings should suffice and the user will not need to configure anything extra.

Gulp users will be familiar with using task as the entity of work to be processed by the build tool. In this plugin the keyword target is used to signify a Gulp task. This is done to prevent accidental clashes with the common Gradle keyword called task.

GulpTask type

build.gradle
myGulpTask {
    target 'clean' (1)
}
1 Set the target task to execute. If nothing is configured, the default Gulp task will be executed.

In some cases the global Gulp configuration might not be suitable and per-task customisations can be done by accessing nodejs, npm and gulp extensions on the task itself. the task will always look at the local extensions first before retrieving values from the global extensions.

build.gradle
myGulpTask {
    gulp {
        gulpFile 'foo/gulpfile.js' (1)
    }
    npm {
        homeDirectory 'foo' (2)
    }
}
1 Override the location of gulpfile.js for this specific task.
2 Use a different NPM setup to run this task.

This shows the flexibility of the plugin and how behaviour can be changed to deal with very specific situations. In most cases you will not need it, but when you need to do something extraordinary, it shuol dbe able to be configured.

Change dependency group

By default Gulp will be added to devDependencies. It is possible to override and make it a default or optional dependency.

build.gradle
gulp {
    installGroup NpmDependencyGroup.OPTIONAL (1)
}
1 See NpmDependencyGroup for valid settings.

Command-line operations

Running Commands from Command-line

Sometimes it is necessary to run commands from the command-line, because the plugin might yet do what your require, or you just want to test something arbitrarily. TH egood news is that you don;t have to have separate installation for this. With the org.ysb33r.nodejs.cmdline plugin you can run node, npm and npx via Gradle and pass all the required arguments. By default this will execute in the NPM home directory.

$ ./gradlew node '--args=--check app/index.js' (1)
$ ./gradlew node --arg=--check' --arg app/index.js (2)
$ ./gradlew npm --args=version (3)
$ ./gradlew npx --args=gulp (4)
1 Run node using --args which accept a list of arguments that are space separated. Not the use of quotes to prevent shell escaping.
2 Run node using multiple --arg options. Each one will be added to the node command-line.
3 Run npm. Both --args and --arg can be used.
4 Run npx. Both --args and --arg can be used.

Customising the command-line

Environmental settings for these commands are controlled from the nodejs and npm project extensions. In the case of npm and npx the npm_config_userconfig and npm_config_globalconfig environmental variables will be set. This configuration should be sufficient for most cases, but should you feel the need to customise you have the following options available from the tasks.

environmentProvider

This setter takes a provider of Map<String,Object> which will set the environment for the command, rather than reading it from

workingDir

This setter take anything than can be converted to File and is then used instead of the npm.homeDirectory setting.

tasks.node { (1)
    workingDir = projectDir (2)
}
1 Works the same for npm and npx. Due to the presence of the npm extension, using tasks.npm will be needed in the Groovy DSL.
2 Override working directory.

Using Wrappers

If the above solutions are not sufficient for you, consider generating wrappers into your project. These wrappers will use the version of node specified in the project’s nodejs extension. They can be checked into your source repository.

Benefits
  • You can run any of these like you would run the original binaries and they would use the versions tied to your project. As such everyone in your team are using the correct versions.

  • You can also update your IDE to use these wrappers instead of using the global installed ones.

  • You don’t have to fiddle with nvm or remember which version to switch to.

Usage
plugins {
  id 'org.ysb33r.nodejs.wrapper' version '{revnumber}'
}

Now just run ./gradlew nodeWrappers.

Developing with Node

Using Gradle to Manage Javascript Projects

Using Gradle to manage full Node.js project development might seem strange to many, but there is actually a great synergy. Gradle can be used to manage the whole ecosystem around the project including:

  • Bootstrapping tools.

  • Managing infrastructure.

  • Controlling pipelines and deployment.

  • Consolidating reports.

  • Managing releases.

  • One of more JavaScript projects can be managed as part of a much larger multi-project build.

Even-though there are a vast range of tools available in the NPM ecosystem, tying everything together can be painful. If this this done correctly Gradle can be transparent to most of your Javascript development team.

Setting it up

Start by adding an empty build.gradle file as well as your settings.gradle file. Remember to add a wrapper as well.

Directory layout
├── build.gradle
├── gradle
│   └── wrapper
│       ├── gradle-wrapper.jar
│       └── gradle-wrapper.properties
├── gradle.properties
├── gradlew
├── gradlew.bat
├── settings.gradle
└── src
    └── node  (1)
        ├── app
        ├── tests
        ├── node_modules
        ├── package.json
        └── package-lock.json
1 Place all of your Javascript code in src/node including files such as .editorconfig, .eslintrc.js etc. In this way your pure Javascript developer can simply point their editor to this folder after checkout.

Edit build.gradle to add the necessary plugin(s).

build.gradle
plugins {
  id 'org.ysb33r.nodejs.dev' version '0.10.0' (1)
  id 'org.ysb33r.nodejs.wrapper' version '0.10.0' (2)
}
1 The dev plugin provides all of the necessary conventions for marrying Javascript development with Gradle management.
2 The use of the wrapper plugin is not required, but is recommended especially for IDE integration.

Apply the org.ysb33r.nodejs.dev plugin will: * Set up npm.homeDirectory to point to src/node * Add packageJsonInstall and syncProjectToPackageJson tasks * Add rules to run NPM scripts defined in the scripts block of package.json.

Running scripts

If you have any scripts defined in package.json you can run them from Gradle by simply doing npmRunTest (for a script called test).

If you need to configure the specific task to pass parameters, then you can add them

npmRunTest {
   cmdArgs '--grep=pattern' (1)
}
1 The arguments will be passed to the script after -- has been passed. Thus this example is the equivalent of npm run test -- --grep=pattern.

Managing Lifecycle in Node.js Projects

Running the assemble task will * Ensure that the project’s version is synchronised to the version in package.json. It will also perform adjustments to ensure that the version conforms to semantic versioning. * Install all necessary packages via NPM.

Controlling package.json format

When updating the package.json file, rewriting might conflict with the coding standards of some teams. For this reason `syncProjectToPackageJson offers some configuration possibilities.

syncProjectToPackageJson {
  forceTwoSpaceIndent = true (1)
  sortOutput = true (2)
}
1 Force a two-space identation when writing out. Default false.
2 Sorts the elements in a similar way to the sort-package-json NPM package.

Advanced Topics

Using Javascript Tools in the Build

Many times people just want to utilise a useful tool that is available from NPM, but they do not want to install a whole raft of global infrastcture just to use the tool. Also they do not want other consumers of their build to have to do the same.

Approach 1: Bootstrapping everything

Let’s assume you want to run Antora with Gradle. You will need to set up some tasks to

  • Create a package.json file on your behalf.

  • Populate it with the required Antora NPM packages.

  • Install them locally within your build space

build.gradle
ext {
    antoraVersion = '2.2' (1)
}

nodejs {
    useSystemPath() (2)
}

npm {
    homeDirectory = "${buildDir}/antora" (3)
}

dependencies {
    npm npmPackage(scope: 'antora', name: 'cli', tag: antoraVersion, type: 'dev') (4)
    npm npmPackage(scope: 'antora', name: 'site-generator-default', tag: antoraVersion, type: 'dev')
}

task initPackageJson(type: org.ysb33r.gradle.nodejs.tasks.NpmPackageJsonInit) (5)

task syncPackageJson(type: org.ysb33r.gradle.nodejs.tasks.SyncProjectToPackageJson) { (6)
    dependsOn initPackageJson (7)
    forceSemver = false  (8)
    packageJsonVersion = '0.0.0'
    packageJsonFile = initPackageJson.packageJsonFileProvider
    npmConfigurations 'npm' (9)
}

task installAntora( type : org.ysb33r.gradle.nodejs.tasks.NpmPackageJsonInstall ) { (10)
    dependsOn syncPackageJson
}
1 Decide on which version of Antora you are going to use.
2 Run Node with the system search path as Gradle otherwise Antora’s packages will not install correctly.
3 For this example we are keeping the source directory clean and running everything inside the build directory in a folder called antora.
4 Define your required Antora packages in the npm configuration. They are specified with type dev as they are purely used for runnign a tool.
5 Create a task that will bootstrap package.json. You don’t need to specify anything extra as this task knowns from the npm extension where to create package.json
6 Create the synchronisation task that will write all of the necessary package definitions into package.json
7 This task can only run if package.json has been created, so we need a dependency.
8 Because this is only running a tool, we do not really care about a properly formatted version.
9 Tell the task to locate all of the packages inside the npm configuration.
10 Create a task that will download and install all of the required NPM packages in the antora/node_modules directory.

If you run ./gradlew installAntora at this point you should see all of the required Antora pacakges along with their transitive dependencies installed in build/antora/node_modules. So what is left now is to add task which can actually execute Antora with different parameters. For this exampe we are only going to do the basic use case of running Antora with its playbook.

build.gradle
task runAntora { (1)
    dependsOn installAntora (2)
    doLast {
        nodejs.exec { (3)
            workingDir npm.homeDirectoryProvider (4)
            script 'node_modules/@antora/cli/bin/antora' (5)
            scriptArgs '../../antora-playbook.yml' (6)
            executable nodejs.executable (7)
        }
    }
}
1 Create a basic task.
2 It needs to have Antora installed before it can run.
3 Use the nodeexec project extension to run node.
4 The working directory must be the same as where your Antora project is located i.e. the NPM home directory.
5 The location of the Antora script relative to the working directory. If you don;t know this, run installAntora first and then inspect the node_modules directory.
6 Provide some paraneters for the task. In this case we are going to point to the location of the Antora playbook file relative to the working directory. If you want to try this example out you can get a demo playbook file.
7 The location of the node executable.

You are now left with just running ./gradlew runAntora.

Approach 2: Supplying a package.json file

If you know enough about Node development you can craft up a package.json file and have it in your project. This will reduce some of the code shown above.

build.gradle
nodejs {
    useSystemPath() (1)
}

task installAntora( type : org.ysb33r.gradle.nodejs.tasks.NpmPackageJsonInstall ) (2)
1 Run Node with the system search path as Gradle otherwise Antora’s packages will not install correctly.
2 Create a task that will download and install all of the required NPM packages in the`node_modules` directory. If you place your package.json file in the project directory you do not need to make any more configuration.
The package.json file does not have to be in the top of your project, but you will need to either copy it somewhere or be prepared to run the project in the folder whether this file exists and then exclude generated artifacts from source control. In this case you will have to configure the NpmPackageJsonInstall task.

You are now left with creating a task to run Antora. This is similar to before except that file locations will be different.

build.gradle
task runAntora {
    dependsOn installAntora
    doLast {
        nodejs.exec {
            script 'node_modules/@antora/cli/bin/antora'
            scriptArgs 'antora-playbook.yml'
            workingDir npm.homeDirectoryProvider
            executable nodejs.executable
        }
    }
}

Run ./gradlew runAntora and your site will be created.

Wrapping NPM packages

If you are familiar with NPM just will know that many packages have some entry point script which can be used as a tool on the command-line. You are probably quite familiar with Gulp too and if you already usedh the Gulp functionality that comes with the Gulp plugin, you might be pondering doing the same approach for your favourite Node tool package.

The process if doing this is relatively straight-forward. It involves creating an extension and a resolver. Once you have these you can continue implementing your own task types.

The following example are shown in Groovy, but you can use Java or Kotlin for your implementation if you wish.

This approach can be used in a gradle script, buildSrc or a standalone plugin.

In 0.1 it was still necessary to create a separate resolver. As from 0.2 this is no longer necessary and you can proceed to just creating the extension.

Creating the extension

Assume for the moment that there is a packaged tool called FooBar which you want to wrap.

Firstly, create an extension class that extends AbstractPackageWrappingExtension.

There are three methods in protected scope that you will need to implement in addition to two contructors

@CompileStatic
class FooBarExtension extends AbstractPackageWrappingExtension {

    FooBarExtension(Project project) { (1)
        super(project)
    }

    FooBarExtension(Task task) { (2)
        super(task,'foobar') (3)
    }

    @Override
    protected String getExtensionName() { (4)
        'name-of-this-extension'
    }

    @Override
    protected String getEntryPoint() { (5)
        'bin/foobar.js'
    }

}
1 Attaches the extension to a project.
2 Attaches the extension to a task
3 It is necessary top pass the name of the project extension, when creating the task extension.
4 The name that this extension will be known by. This is used by the superclass to resolve project extensions when the extensions are attached to a task. (This functionality might be removed in future).
5 This is the entrypoint script file and must be specified relative to the package folder after installation.

You can now add methods to your extension as if necessary to reflect the functionality of the wrapped package. For instance the GulpExtension class adds gulpFile and requires to deal with the --gulpfile and --requires parameters of gulp.js.

If you add methods where the returned values can be overriden in a task in a similar fashion to what can be done in NpmTask and GulpTask, you will need to add some logic to handle those cases. Here is an example of how it is being done in GulpExtension to retrieve the value for gulpFile.

GulpExtension.groovy
File getGulpFile() {
    if (task) { (1)
        this.gulpFile != null ? projectOperations.file(this.gulpFile) : ((GulpExtension) projectExtension).gulpFile
    } else {
        projectOperations.file(this.gulpFile)
    }
}
1 If this is attached to a task, check if the value is set. Use it if it is, otherwise defer to the project extension. If this is a project extension use the local value.

Create a task type

In some cases you might want to also add a task type, similar to GulpTask. One approach is to extend AbstractNodeBaseTask.

FooTask.groovy
@CompileStatic
class FooTask extends AbstractNodeBaseTask {

  FooTask() {
    super()
    fooExtension = (FooExtension)(extensions.create('foo',FooExtension,this))
  }

  @TaskAction
  void exec() {
        NodeJSExecSpec execSpec = createExecSpec() (1)

        execSpec.script fooExtension.resolvedExecutable.executable.absolutePath (2)

        /* Configure execution specification against any properties */

        runExecSpec(execSpec) (3)

  }

  private FooExtension fooExtension
}
1 Base class has ability to create and pre-configure an execition specification.
2 Resolve the script from the resolver.
3 Execute the execution specification.

Tips and tricks

Overriding Node Repository URI

There are circumstances where you might want to download Node distributions from elsewhere besides the standard repository. Simply set the system property org.ysb33r.gradle.nodejs.uri to the alternative URI.