This post was written by Manuel Pais, co-author of Team Guide to Software Releasability.

Continuous Integration (CI) enables you to have a reliable, repeatable, automated process to prepare your site for release, removing manual, error prone steps. With relative ease, you can add in automated checks, both generic static analysis and moving onto custom automated tests to check specific functionality. A common misconception is that you only need a CI build if you are actually compiling code, and that it doesn’t make sense if you are just creating a static website. However, there are a large number of benefits that can be gained with CI for static websites with very little effort.

In this post, we cover:

Validating HTML using htmllint – checking for things like missing mandatory elements, duplicate id’s, checking the site is accessible to screen readers, or even best practice suggestions
Validating JavaScript – checking for things like undeclared variables, missing semi-colons and potential memory leaks, and minifying large JavaScript files
Tracking page load speed using PageSpeed – checking early for any obvious performance problems with HTML+JavaScript

These techniques don’t only apply to pure static html websites, this can also be applied to sites generated with tools such as Jekyll. And while a CMS can manage a lot of these things for you, that can be overkill for a relatively simple site due to the added complexity such as the promotion through environments.

This post walks through some small steps you can take to increase quality and remove manual, error prone processes and testing.

Using Grunt as a task runner

The tools we are using below work very well with Grunt – a popular javascript task runner that is designed for automating repetitive tasks in software development. Grunt uses Node.js, but that is just a simple install via binary, or via a package manager. Once Node.js is installed, it’s recommended that you update the Node Package Manager (NPM) to make sure you’ve got access to the latest and greatest tools:

npm install npm -g

(Don’t forget to add sudo on the front if you are running on a *nix system.)

Add a new file package.json to your project, with the following content:

{
  "name": "demo-site",
  "version": "0.1.0",
  "description": "awesome demo site",
  "repository": "not yet available",
  "license": "MIT"
}

Then run:

npm install -g grunt-cli
npm install grunt --save-dev

which will install Grunt and the Grunt command line tool.

Add another file Gruntfile.js with the following content:

module.exports = function(grunt) {

  // Project configuration.
  grunt.initConfig({
    pkg: grunt.file.readJSON('package.json'),
  });

  // Default task(s).
  grunt.registerTask('default', []);
};

Running grunt in your project directory should run successfully:

localhost:demo-site$ grunt
Done, without errors.

Now that the plumbing is out of the way, we can get on with the much more interesting parts.

Validating HTML using htmllint

A linter is a program that will analyse your source code (HTML in this case) and validate it against a bunch of rules. You can consider it similar to the way a compiler makes sure your code makes sense before turning it into an compiled binary.

Linting can help a lot with enforcing standards, and especially can help a lot around accessibility and preventing use of deprecated features that some browsers no longer support.

Install grunt-htmllint by running:

npm install grunt-htmllint --save-dev

and modify your Gruntfile.js to look like:

module.exports = function(grunt) {

  // Project configuration.
  grunt.initConfig({
    pkg: grunt.file.readJSON('package.json'),
    htmllint: {
      options: { },
      src: [
        '_site/**/*.html'
      ]
    },
  });

  grunt.loadNpmTasks('grunt-htmllint');

  // Default task(s).
  grunt.registerTask('default', ['htmllint']);
};

Running grunt now tells us about a bunch of issues, and stops, showing an error:

localhost:demo-site$ grunt
Running "htmllint:src" (htmllint) task
>> _site/about/index.html: (E036), line 4, col 1, indenting spaces must be used in groups of 4
>> _site/about/index.html: (E036), line 5, col 1, indenting spaces must be used in groups of 4 
>> _site/about/index.html: (E036), line 6, col 1, indenting spaces must be used in groups of 4 
>> _site/about/index.html: (E036), line 7, col 1, indenting spaces must be used in groups of 4 
>> _site/about/index.html: (E036), line 9, col 1, indenting spaces must be used in groups of 4 

... 

>> _site/jekyll/update/2015/06/16/welcome-to-jekyll.html: (E011), line 134, col 18, value must match the format: underscore 
>> encountered 264 errors in total 
>> 3 file(s) had lint error out of 3 file(s). (skipped 0 files) 
Warning: Task "htmllint:src" failed. Use --force to continue. 

Aborted due to warnings.

There are many options available – some of which will make sense for your project, and some which won’t. Start small, and extend as you can. For now, we’ll disable the rules that are causing the issues by including the options in Gruntfile.js

module.exports = function(grunt) {

  // Project configuration.
  grunt.initConfig({
    pkg: grunt.file.readJSON('package.json'),
    htmllint: {
      options: {
        'indent-width': false,
        'id-class-style': false,
        'attr-name-style': false
      },
      src: [
        '_site/**/*.html'
      ]
    },
  });

  grunt.loadNpmTasks('grunt-htmllint');

  // Default task(s).
  grunt.registerTask('default', ['htmllint']);
};

Validating JavaScript

Now that we’ve got our HTML all validated and looking good, we can now look at linting our JavaScript, in this case, using grunt-contrib-jshint.

Install by running the command:

npm install grunt-contrib-jshint --save-dev

Once installed, load the tasks in the Gruntfile.js with the following changes:

module.exports = function(grunt) {

  // Project configuration.
  grunt.initConfig({
    pkg: grunt.file.readJSON('package.json'),
    htmllint: {
      options: {
        'indent-width': false,
        'id-class-style': false,
        'attr-name-style': false,
      },
      src: [
        '_site/**/*.html'
      ]
    },
    jshint: {
      src: ['js/main.js'],
    },
  });

  grunt.loadNpmTasks('grunt-htmllint');
  grunt.loadNpmTasks('grunt-contrib-jshint');

  // Default task(s).
  grunt.registerTask('default', ['jshint', 'htmllint']);
};

Running grunt now shows us that we forgot a semicolon:

localhost:demo-site$ grunt
Running "jshint:src" (jshint) task

   js/main.js
     15 |        method = methods[length]
                                         ^ Missing semicolon.

>> 1 error in 1 file
Warning: Task "jshint:src" failed. Use --force to continue.

Aborted due to warnings.

Minification

Now that we’ve got a rough “build” process, we can start extending it to add some optimisations that will make the user experience much better.

Assuming that we have all of our javascript in one file (if not, we can easily make this happen with grunt-contrib-concat), we can now add some minification, to ensure our javascript is as small as possible so that it downloads quickly.

Note: Concatenation and minification only make sense when using HTTP 1.x. As the web moves to HTTP 2.0, some of these techniques are considered an anti-pattern.

Install the grunt-contrib-uglify module:

npm install grunt-contrib-uglify --save-dev

and update the Gruntfile.js to setup the task:

module.exports = function(grunt) {

  // Project configuration.
  grunt.initConfig({
    pkg: grunt.file.readJSON('package.json'),
    htmllint: {
      options: {
        'indent-width': false,
        'id-class-style': false,
        'attr-name-style': false,
      },
      src: [
        '_site/**/*.html'
      ]
    },
    jshint: {
      src: ['js/main.js'],
    },
    uglify: {
      js: {
        files: {
          'js/combined.js': ['js/main.js']
        }
      }
    },
  });

  grunt.loadNpmTasks('grunt-htmllint');
  grunt.loadNpmTasks('grunt-contrib-jshint');
  grunt.loadNpmTasks('grunt-contrib-uglify');

  // Default task(s).
  grunt.registerTask('default', ['jshint', 'htmllint', 'uglify']);
};

From here, we could use something like grunt-processhtml to re-write our html to refer to the new javascript file name.

PageSpeed

Google PageSpeed is a rather nice technology that can run performance tests against your site, and give you valuable feedback about what could be improved to give a better experience for your users. Normally, this would only be available on a publicly available website, but with a fancy bit of technology called NGrok we can do it during local testing as well.

Install the grunt-pagespeed, grunt-contrib-connect and ngrok modules:

npm install grunt-pagespeed --save-dev
npm install grunt-contrib-connect --save-dev
npm install ngrok@0.1.99 --save-dev

Note: ngrok is fixed at version 0.1.99 because of issue #248 (open at the time of writing). Once that’s closed you should be able to use the latest version (npm install ngrok --save-dev) but beware ngrok 2.0 onwards requires an authtoken.

Then extend the Gruntfile.js to add a new custom task:

var ngrokConnector = require('ngrok');
module.exports = function(grunt) {

  // Project configuration.
  grunt.initConfig({
    pkg: grunt.file.readJSON('package.json'),
    connect: {
      server: {
        options: {
          port: 40918,
          base: '_site'
        }
      }
    },
    htmllint: {
      options: {
        'indent-width': false,
        'id-class-style': false,
        'attr-name-style': false,
      },
      src: [
        '_site/**/*.html'
      ]
    },
    jshint: {
      src: ['js/main.js'],
    },
    uglify: {
      js: {
        files: {
          'js/combined.js': ['js/main.js']
        }
      }
    },
    pagespeed: {
      options: {
        nokey: true
      },
      desktop: {
        options: {
          locale: "en_GB",
          strategy: "desktop",
          threshold: 80
        }
      },
    }
  });

  grunt.loadNpmTasks('grunt-htmllint');
  grunt.loadNpmTasks('grunt-contrib-jshint');
  grunt.loadNpmTasks('grunt-contrib-uglify');
  grunt.loadNpmTasks('grunt-pagespeed');
  grunt.loadNpmTasks('grunt-contrib-connect');

  grunt.registerTask('pagespeed-via-ngrok', 'Run Google PageSpeed via ngrok', function() {
    var done = this.async();
    var port = 40918;

    ngrokConnector.connect(port, function(err, url) {
      if (err !== null) {
        grunt.fail.fatal(err);
        return done();
      }
      grunt.config.set('pagespeed.options.url', url);
      grunt.task.run('pagespeed');
      done();
    });
  });

  grunt.registerTask('perf-tests', ['connect', 'pagespeed-via-ngrok'])

  // Default task(s).
  grunt.registerTask('default', ['jshint', 'htmllint', 'uglify']);
};

Note we haven’t modified the default task – we’ve made it an explicit call, as its more of an integration test, rather than a unit test:

localhost:demo-site$ grunt perf-tests

And we get output similar to:

Running "connect:server" (connect) task
Started connect web server on http://localhost:40198

Running "pagespeed-via-ngrok" task

Running "pagespeed:desktop" (pagespeed) task

--------------------------------------------------------

URL:       63a837fa.ngrok.com
Strategy:  desktop
Speed:     96

CSS size                                   | 8.76 kB
HTML size                                  | 5.68 kB
CSS resources                              | 1
Hosts                                      | 1
Resources                                  | 2
Total size of request bytes sent           | 92 B

Enable GZIP compression                    | 0.66
Main resource server response time         | 0.57
Minify CSS                                 | 0.23
Minimize render blocking resources         | 2

--------------------------------------------------------

Done, without errors.

We can easily adjust the threshold up and down to find the appropriate level for each project.

Tying it all together

Now that we’ve got all of these individual pieces, we can create a CI pipeline to tie them all together, giving us a all the benefits of these checks on every commit to source control. In our example here, we are using GoCD with a final step to FTP to production.

Conclusion

Hopefully this post has given a good overview of how automated build and testing procedures are easily achievable, even with a static website. We can gain huge benefits from the repeatable, automated and tested approach to releasing the site, with very little effort.

Thanks to James Cryer for his original work on using grunt, ngrok and pagespeed for local testing.

Show me the code

For those who want to dig into the code a bit more, we’ve created a GitHub repo that works through these examples. Have a play and let us know what you think.