Coding

Gulp.js – an AMAZING build system!


Every once in a while, I learn something new that makes me just really happy to be a programmer. Recently, I started building a single-page javascript game, and discovered gulp.js, a build system where you write your build scripts in javascript. Multiple separate javascript, LESS, and HTML files, all automatically compiled and concatenated when they change, and auto-built into a single, deployable directory, and all from a simple, easy-to-understand build script. It feels almost magical, definitely futuristic, and I’m going to share it with you here.

First, we have to make sure everything is set up to begin. You’re going to want to install Node.js, since gulp is a node script. Next, you’ll want to install NPM – the official package manager for Node.js. Finally, install gulp:

npm install -g gulp

Next, we need to set up our project, and we’re going to do it right. Begin with a “package.json” file, with all the details about your project:

{
    "name": "sample",
    "version": "0.1.0",
    "author": "Dan Hulton <dan@danhulton.com>",
    "description": "A sample project.",
    "private": true,
    "repository": {
        "type": "git",
        "url": "https://github.com/DanHulton/sample-project.git"
    },
    "devDependencies": {
    }
}  

The version number is important, since we’re going to be using that in our gulp file later for some smart stuff, as is the “private” value. Without it, it’s possible you could accidentally “npm publish” your project, which would add it to the global repository of npm projects. “devDependencies” is empty for now, but we’ll fix that.

You’ll need to create a gulpfile as well. This is your build script. We’ll start with a simple one:

/* jslint node: true */
"use strict";

var gulp   = require('gulp');
var concat = require('gulp-concat');

// Package information, including version
var pkg = require('./package.json');

var info = {};

info.src = {
    js : 'src/js/**/*.js',
};

info.dest = {
    js : 'build',
};

// Build JS files
gulp.task('js', function () {
    return gulp.src(info.src.js)
        .pipe(concat('all-' + pkg.version + '.js'))
        .pipe(gulp.dest(info.dest.js));
});

We’re doing a lot of interesting things here with very little code, so let me go over them.

On lines 1&2, we’re giving information to jshint, a tool that you should really begin using. If you can dig up a plugin for your editor of choice, it will constantly be telling you when you’re engaging in low-quality javascript practices (comparing against null with “==”, undeclared variables, etc). Here, we’re telling it to be strict about the warnings it gives us, and to assume that we’re using node.js and expect certain global variables to be defined because of it.

In the next two lines, we’re requiring the various gulp modules we’ll be using – “gulp” itself, and “gulp-concat”, which we’ll use to concatenate our javascript files. By the way, we’ll need to install those locally for this project, which we can do with:

npm install --save-dev gulp
npm install --save-dev gulp-concat

This will create a “node_modules” folder in your project containing both of these modules, as well as modify the “devDependencies” object in your “package.json”. This is good, because if you ever need this project to work on another machine (likely, if you plan on working with more than one person), all they’ll need to do is run “npm install” in the root folder and it will automatically fetch and install all the build dependencies for you. (That is, if you don’t commit your “node-modules”, but why wouldn’t you?)

On line 8, we’re including the “package.json” file directly, which we use later.

Lines 10-18 describe the various directories important to our build process, and actually, let’s set those up now:

Basic folder setup

Create an empty “build” folder, and a “src” folder containing a “js” folder containing two javascript files, named “one.js” and “two.js” respectively.

All you need in each file is a simple console.log:

console.log("One.");
console.log("Two!");

Back to our gulpfile, though. On line 21, we’re creating a task called “js”. Line 22 begins the gulp magic by sourcing all the files we defined in the “info.src.js” pattern. In this case, that’s everything in “src/js” and all its subfolders.

Line 23 uses the gulp “pipe()” function to pass the results of the previous action to a new one, in this case, the “concat” plugin we installed earlier. This concatenates all the files that were sourced into one, and we provide the new filename as a parameter. Notice how we’re referencing the version number from the package from before. The resulting filename will be “all-0.1.0.js”.

Finally, line 24 outputs the resulting file to the path defined in “info.dest.js” – the build directory.

So check out the magic! Go to your new project’s root folder and type in:

gulp js

This should run the “js” task as defined in your gulpfile, which will concatenate the two javascript files we created and dump them in your build directory. Neat, huh?

But let’s see if we can’t make this a little more useful, hm? Our javascript file is going to live inside some HTML file somewhere, why not see if we can get it automatically included? It’s a lot easier than you might think.

/* jslint node: true */
"use strict";

var gulp     = require('gulp');
var concat   = require('gulp-concat');
var inject   = require('gulp-inject');
var clean    = require('gulp-clean');
var sequence = require('run-sequence');

// Package information, including version
var pkg = require('./package.json');

var info = {};

info.src = {
    html : 'src/html/index.html',
    js   : 'src/js/**/*.js',
};

info.dest = {
    html : 'build',
    js   : 'build',
};

// Clean build folder
gulp.task('clean', function () {
  return gulp.src('build/*', {read: false})
    .pipe(clean());
});

// Build JS files
gulp.task('js', function () {
    return gulp.src(info.src.js)
        .pipe(concat('all-' + pkg.version + '.js'))
        .pipe(gulp.dest(info.dest.js));
});

// Build HTML files
gulp.task('html', function() {
    return gulp.src(info.src.html)
        .pipe(inject(gulp.src(info.dest.js + '/*.js', { read: false }), {
            addRootSlash: false,
            ignorePath: '/' + info.dest.html + '/'
        }))
        .pipe(gulp.dest(info.dest.html));
});

// Full build - clean build folder, build js, build html
gulp.task('build', function(callback) {
  sequence(
        'clean',
        'js',
        'html',
        callback
    );
});

gulp.task('default', ['build']);

Okay, so that’s a lot of new code, but it’s all pretty easy to understand, really. Let me walk you through it.

Lines 6-8 require in some new packages that we’ll need, and hey, let’s just do that at the command line like before, too:

npm install --save-dep gulp-inject
npm install --save-dep gulp-clean
npm install --save-dep run-sequence

Lines 16 & 21 set up source and destination folders for our new HTML files. While we’re at it, why don’t we create that new HTML folder and an “index.html” file inside of it:

<html>
<body>
    <h1>Sample Project</h1>

    <!-- inject:js -->
    <!-- any *.js files among your sources will go here as: <script src="FILE"></script> -->
    <!-- endinject -->
</body>
</html>

Lines 26-29 create a new task called “clean”, which sources all the files in our build folder, and pipes them into the “clean()” plugin, which just deletes them. This ensures every new build starts with fresh files, which is useful when incrementing version numbers.

Lines 39-46 load our HTML file and inject our generated javascript file inside of it. It’s not as clean a plugin as some of the others, and requires a bunch of extra options to get it to work correctly, but it gets the job done.

Lines 49-56 create a new task called “build”, which runs our clean, js, and html tasks in sequence to build the project.

Finally, line 58 creates a new task called “default” which just runs the “build” task. Why create a task that’s just a shortcut to another task? Well, gulp expects a task called “default”, which it runs when there’s no other parameters sent via the command line. So let’s try it now:

gulp

You should get two files in your build folder:

<html>
<body>
    <h1>Sample Project</h1>

    <!-- inject:js -->
    <script src="all-0.1.0.js"></script>
    <!-- endinject -->
</body>
</html>
console.log("One.");
console.log("Two!");

Ta-da! Add more javascript files, play around with the HTML, go nuts! Every time you run gulp, your changes will be reflected in the build files.

Now, there’s still a lot more you can do with gulp, actually. For example, we could add a LESS preprocessor so we can split up our CSS and write nice, formatted styles. We could minify our JS. We could include images that get copied to the build folder. Most interestingly and importantly, we could set it up so that gulp constantly watches our source folders and automatically runs the build step whenever anything changes!

And these are all small changes, too! Small improvements! But this is already a big blog entry, and I’m loathe to make it much larger. I’ll write a follow-up soon that details how to modify our existing skeleton and flesh it out some. In the meantime, don’t hesitate to browse the npm repository for gulp modules. There are a lot of interesting and useful tools you can add to you build process!

Coding
Code faster with simple Sublime Text improvements
Coding
Rage-quit support for fish shell
  • Shona

    ShonaShona

    Author

    Hi,
    Fantastic article…
    I need some help from you, how do I add del dependency in gulp.
    Example : var del = require(‘del’);
    Which dependency is needed to access the example statement.


  • Dan Hulton

    Dan HultonDan Hulton

    Author

    Well, your code there looks fine, assuming you have the del package installed.

    Outside of that, I’m not sure I understand your problem adequately enough to help you out there.


  • ibrahim

    ibrahimibrahim

    Author

    Hi. How to implement gulp-inject to add script tags into index.html every time we add a new file?


  • Casey

    CaseyCasey

    Author

    You have a few places where it says “–save-dep” instead of “–save-dev”.